fix(gateway): stage safe local MEDIA files before delivery - #38108
fix(gateway): stage safe local MEDIA files before delivery#38108GodsBoy wants to merge 1 commit into
Conversation
Agents commonly produce deliverables in a working directory (proposal.docx, report.pdf) rather than inside a Hermes cache. When the gateway runs as root, the whole /root home is on the system denylist, so validate_media_delivery_path drops the path and the user gets a chat reply that claims an attachment with nothing attached. Copying the same file under ~/.hermes/cache/documents/ made it deliver. stage_media_delivery_path copies a safe local deliverable into the media cache matching its extension, then returns the staged path so normal MEDIA delivery proceeds. filter_media_delivery_paths and filter_local_delivery_paths now try direct validation first, then the staging fallback, then rejection. The denylist is preserved. Staging refuses credential/system directories via the existing _media_delivery_denied_paths, refuses credential filenames (.netrc, id_rsa, *.pem, ...), resolves symlinks before any check, and in strict mode only stages freshly produced files. The running user's own home is the one allowed exception to the broad denied-prefix, which is what lets a root-run gateway deliver /root/work/proposal.docx while /root/.ssh stays blocked. No arbitrary working directory is allowlisted. Adds 21 tests covering staging, denylist preservation, collision-safe names, strict-mode recency, and the filter integration.
Root-run gateways have $HOME=/root, which is on the MEDIA system-path denylist, so the gateway silently dropped agent-generated deliverables under /root (e.g. /root/work/proposal.docx) — the user got a 'here is your file' reply with nothing attached. _path_under_denied_prefix now treats the running user's own home as deliverable: the home tree itself is no longer denied, while the more-specific denied paths inside it (~/.ssh, ~/.aws, ~/.hermes/.env, auth.json, config.yaml) stay blocked because they are separate denylist entries. The exception only matches when the denied prefix IS $HOME, so a non-root gateway still can't deliver another user's home. Diagnosis, reproduction, and the failing-case analysis are from @GodsBoy (#38108 / #38106). Implemented here as the minimal denylist fix rather than a staging/copy subsystem. Co-authored-by: GodsBoy <dhuysamen@gmail.com>
|
Thanks for the diagnosis and the live Telegram repro — that's what made this actionable. We shipped the fix as #39063 (merged to main as Your authorship is preserved via |
Root-run gateways have $HOME=/root, which is on the MEDIA system-path denylist, so the gateway silently dropped agent-generated deliverables under /root (e.g. /root/work/proposal.docx) — the user got a 'here is your file' reply with nothing attached. _path_under_denied_prefix now treats the running user's own home as deliverable: the home tree itself is no longer denied, while the more-specific denied paths inside it (~/.ssh, ~/.aws, ~/.hermes/.env, auth.json, config.yaml) stay blocked because they are separate denylist entries. The exception only matches when the denied prefix IS $HOME, so a non-root gateway still can't deliver another user's home. Diagnosis, reproduction, and the failing-case analysis are from @GodsBoy (NousResearch#38108 / NousResearch#38106). Implemented here as the minimal denylist fix rather than a staging/copy subsystem. Co-authored-by: GodsBoy <dhuysamen@gmail.com>
Root-run gateways have $HOME=/root, which is on the MEDIA system-path denylist, so the gateway silently dropped agent-generated deliverables under /root (e.g. /root/work/proposal.docx) — the user got a 'here is your file' reply with nothing attached. _path_under_denied_prefix now treats the running user's own home as deliverable: the home tree itself is no longer denied, while the more-specific denied paths inside it (~/.ssh, ~/.aws, ~/.hermes/.env, auth.json, config.yaml) stay blocked because they are separate denylist entries. The exception only matches when the denied prefix IS $HOME, so a non-root gateway still can't deliver another user's home. Diagnosis, reproduction, and the failing-case analysis are from @GodsBoy (NousResearch#38108 / NousResearch#38106). Implemented here as the minimal denylist fix rather than a staging/copy subsystem. Co-authored-by: GodsBoy <dhuysamen@gmail.com>
Root-run gateways have $HOME=/root, which is on the MEDIA system-path denylist, so the gateway silently dropped agent-generated deliverables under /root (e.g. /root/work/proposal.docx) — the user got a 'here is your file' reply with nothing attached. _path_under_denied_prefix now treats the running user's own home as deliverable: the home tree itself is no longer denied, while the more-specific denied paths inside it (~/.ssh, ~/.aws, ~/.hermes/.env, auth.json, config.yaml) stay blocked because they are separate denylist entries. The exception only matches when the denied prefix IS $HOME, so a non-root gateway still can't deliver another user's home. Diagnosis, reproduction, and the failing-case analysis are from @GodsBoy (NousResearch#38108 / NousResearch#38106). Implemented here as the minimal denylist fix rather than a staging/copy subsystem. Co-authored-by: GodsBoy <dhuysamen@gmail.com>
Root-run gateways have $HOME=/root, which is on the MEDIA system-path denylist, so the gateway silently dropped agent-generated deliverables under /root (e.g. /root/work/proposal.docx) — the user got a 'here is your file' reply with nothing attached. _path_under_denied_prefix now treats the running user's own home as deliverable: the home tree itself is no longer denied, while the more-specific denied paths inside it (~/.ssh, ~/.aws, ~/.hermes/.env, auth.json, config.yaml) stay blocked because they are separate denylist entries. The exception only matches when the denied prefix IS $HOME, so a non-root gateway still can't deliver another user's home. Diagnosis, reproduction, and the failing-case analysis are from @GodsBoy (NousResearch#38108 / NousResearch#38106). Implemented here as the minimal denylist fix rather than a staging/copy subsystem. Co-authored-by: GodsBoy <dhuysamen@gmail.com>
Root-run gateways have $HOME=/root, which is on the MEDIA system-path denylist, so the gateway silently dropped agent-generated deliverables under /root (e.g. /root/work/proposal.docx) — the user got a 'here is your file' reply with nothing attached. _path_under_denied_prefix now treats the running user's own home as deliverable: the home tree itself is no longer denied, while the more-specific denied paths inside it (~/.ssh, ~/.aws, ~/.hermes/.env, auth.json, config.yaml) stay blocked because they are separate denylist entries. The exception only matches when the denied prefix IS $HOME, so a non-root gateway still can't deliver another user's home. Diagnosis, reproduction, and the failing-case analysis are from @GodsBoy (#38108 / #38106). Implemented here as the minimal denylist fix rather than a staging/copy subsystem. Co-authored-by: GodsBoy <dhuysamen@gmail.com>
Root-run gateways have $HOME=/root, which is on the MEDIA system-path denylist, so the gateway silently dropped agent-generated deliverables under /root (e.g. /root/work/proposal.docx) — the user got a 'here is your file' reply with nothing attached. _path_under_denied_prefix now treats the running user's own home as deliverable: the home tree itself is no longer denied, while the more-specific denied paths inside it (~/.ssh, ~/.aws, ~/.hermes/.env, auth.json, config.yaml) stay blocked because they are separate denylist entries. The exception only matches when the denied prefix IS $HOME, so a non-root gateway still can't deliver another user's home. Diagnosis, reproduction, and the failing-case analysis are from @GodsBoy (NousResearch#38108 / NousResearch#38106). Implemented here as the minimal denylist fix rather than a staging/copy subsystem. Co-authored-by: GodsBoy <dhuysamen@gmail.com>
Root-run gateways have $HOME=/root, which is on the MEDIA system-path denylist, so the gateway silently dropped agent-generated deliverables under /root (e.g. /root/work/proposal.docx) — the user got a 'here is your file' reply with nothing attached. _path_under_denied_prefix now treats the running user's own home as deliverable: the home tree itself is no longer denied, while the more-specific denied paths inside it (~/.ssh, ~/.aws, ~/.hermes/.env, auth.json, config.yaml) stay blocked because they are separate denylist entries. The exception only matches when the denied prefix IS $HOME, so a non-root gateway still can't deliver another user's home. Diagnosis, reproduction, and the failing-case analysis are from @GodsBoy (NousResearch#38108 / NousResearch#38106). Implemented here as the minimal denylist fix rather than a staging/copy subsystem. Co-authored-by: GodsBoy <dhuysamen@gmail.com>
…HOME (#54060) Generated images under a profile gateway's cache (profiles/<name>/cache/ images/...) were silently dropped from Telegram/Discord delivery when HERMES_HOME is symlinked under a denied prefix (e.g. /opt/data -> /root/.hermes) and $HOME is not that prefix. The resolved path lands under /root (a system denylist prefix), the root-home exception only fires when the denied prefix IS $HOME, and the static safe-roots list only covers the active HERMES_HOME's top-level cache — not per-profile cache dirs. Both gates fail, so validate_media_delivery_path returns None and the gateway logs 'Skipping unsafe MEDIA directive path'. _media_delivery_allowed_roots() now also enumerates per-profile cache roots (<root>/profiles/*/cache/{images,audio,videos,documents, screenshots}) at check time. Allowlist match runs before the denylist, so the profile artifact delivers regardless of the /root interaction; profile-dir credentials (auth.json) stay blocked since they aren't under a cache subdir. Reopened regression of #34485/#38108, neither of which covered the profile-scoped symlink case. Fixes #31733.
…HOME (NousResearch#54060) Generated images under a profile gateway's cache (profiles/<name>/cache/ images/...) were silently dropped from Telegram/Discord delivery when HERMES_HOME is symlinked under a denied prefix (e.g. /opt/data -> /root/.hermes) and $HOME is not that prefix. The resolved path lands under /root (a system denylist prefix), the root-home exception only fires when the denied prefix IS $HOME, and the static safe-roots list only covers the active HERMES_HOME's top-level cache — not per-profile cache dirs. Both gates fail, so validate_media_delivery_path returns None and the gateway logs 'Skipping unsafe MEDIA directive path'. _media_delivery_allowed_roots() now also enumerates per-profile cache roots (<root>/profiles/*/cache/{images,audio,videos,documents, screenshots}) at check time. Allowlist match runs before the denylist, so the profile artifact delivers regardless of the /root interaction; profile-dir credentials (auth.json) stay blocked since they aren't under a cache subdir. Reopened regression of NousResearch#34485/NousResearch#38108, neither of which covered the profile-scoped symlink case. Fixes NousResearch#31733.
Root-run gateways have $HOME=/root, which is on the MEDIA system-path denylist, so the gateway silently dropped agent-generated deliverables under /root (e.g. /root/work/proposal.docx) — the user got a 'here is your file' reply with nothing attached. _path_under_denied_prefix now treats the running user's own home as deliverable: the home tree itself is no longer denied, while the more-specific denied paths inside it (~/.ssh, ~/.aws, ~/.hermes/.env, auth.json, config.yaml) stay blocked because they are separate denylist entries. The exception only matches when the denied prefix IS $HOME, so a non-root gateway still can't deliver another user's home. Diagnosis, reproduction, and the failing-case analysis are from @GodsBoy (NousResearch#38108 / NousResearch#38106). Implemented here as the minimal denylist fix rather than a staging/copy subsystem. Co-authored-by: GodsBoy <dhuysamen@gmail.com>
…HOME (NousResearch#54060) Generated images under a profile gateway's cache (profiles/<name>/cache/ images/...) were silently dropped from Telegram/Discord delivery when HERMES_HOME is symlinked under a denied prefix (e.g. /opt/data -> /root/.hermes) and $HOME is not that prefix. The resolved path lands under /root (a system denylist prefix), the root-home exception only fires when the denied prefix IS $HOME, and the static safe-roots list only covers the active HERMES_HOME's top-level cache — not per-profile cache dirs. Both gates fail, so validate_media_delivery_path returns None and the gateway logs 'Skipping unsafe MEDIA directive path'. _media_delivery_allowed_roots() now also enumerates per-profile cache roots (<root>/profiles/*/cache/{images,audio,videos,documents, screenshots}) at check time. Allowlist match runs before the denylist, so the profile artifact delivers regardless of the /root interaction; profile-dir credentials (auth.json) stay blocked since they aren't under a cache subdir. Reopened regression of NousResearch#34485/NousResearch#38108, neither of which covered the profile-scoped symlink case. Fixes NousResearch#31733.
…HOME (NousResearch#54060) Generated images under a profile gateway's cache (profiles/<name>/cache/ images/...) were silently dropped from Telegram/Discord delivery when HERMES_HOME is symlinked under a denied prefix (e.g. /opt/data -> /root/.hermes) and $HOME is not that prefix. The resolved path lands under /root (a system denylist prefix), the root-home exception only fires when the denied prefix IS $HOME, and the static safe-roots list only covers the active HERMES_HOME's top-level cache — not per-profile cache dirs. Both gates fail, so validate_media_delivery_path returns None and the gateway logs 'Skipping unsafe MEDIA directive path'. _media_delivery_allowed_roots() now also enumerates per-profile cache roots (<root>/profiles/*/cache/{images,audio,videos,documents, screenshots}) at check time. Allowlist match runs before the denylist, so the profile artifact delivers regardless of the /root interaction; profile-dir credentials (auth.json) stay blocked since they aren't under a cache subdir. Reopened regression of NousResearch#34485/NousResearch#38108, neither of which covered the profile-scoped symlink case. Fixes NousResearch#31733.
…HOME (NousResearch#54060) Generated images under a profile gateway's cache (profiles/<name>/cache/ images/...) were silently dropped from Telegram/Discord delivery when HERMES_HOME is symlinked under a denied prefix (e.g. /opt/data -> /root/.hermes) and $HOME is not that prefix. The resolved path lands under /root (a system denylist prefix), the root-home exception only fires when the denied prefix IS $HOME, and the static safe-roots list only covers the active HERMES_HOME's top-level cache — not per-profile cache dirs. Both gates fail, so validate_media_delivery_path returns None and the gateway logs 'Skipping unsafe MEDIA directive path'. _media_delivery_allowed_roots() now also enumerates per-profile cache roots (<root>/profiles/*/cache/{images,audio,videos,documents, screenshots}) at check time. Allowlist match runs before the denylist, so the profile artifact delivers regardless of the /root interaction; profile-dir credentials (auth.json) stay blocked since they aren't under a cache subdir. Reopened regression of NousResearch#34485/NousResearch#38108, neither of which covered the profile-scoped symlink case. Fixes NousResearch#31733.
Root-run gateways have $HOME=/root, which is on the MEDIA system-path denylist, so the gateway silently dropped agent-generated deliverables under /root (e.g. /root/work/proposal.docx) — the user got a 'here is your file' reply with nothing attached. _path_under_denied_prefix now treats the running user's own home as deliverable: the home tree itself is no longer denied, while the more-specific denied paths inside it (~/.ssh, ~/.aws, ~/.hermes/.env, auth.json, config.yaml) stay blocked because they are separate denylist entries. The exception only matches when the denied prefix IS $HOME, so a non-root gateway still can't deliver another user's home. Diagnosis, reproduction, and the failing-case analysis are from @GodsBoy (NousResearch#38108 / NousResearch#38106). Implemented here as the minimal denylist fix rather than a staging/copy subsystem. Co-authored-by: GodsBoy <dhuysamen@gmail.com>
…HOME (NousResearch#54060) Generated images under a profile gateway's cache (profiles/<name>/cache/ images/...) were silently dropped from Telegram/Discord delivery when HERMES_HOME is symlinked under a denied prefix (e.g. /opt/data -> /root/.hermes) and $HOME is not that prefix. The resolved path lands under /root (a system denylist prefix), the root-home exception only fires when the denied prefix IS $HOME, and the static safe-roots list only covers the active HERMES_HOME's top-level cache — not per-profile cache dirs. Both gates fail, so validate_media_delivery_path returns None and the gateway logs 'Skipping unsafe MEDIA directive path'. _media_delivery_allowed_roots() now also enumerates per-profile cache roots (<root>/profiles/*/cache/{images,audio,videos,documents, screenshots}) at check time. Allowlist match runs before the denylist, so the profile artifact delivers regardless of the /root interaction; profile-dir credentials (auth.json) stay blocked since they aren't under a cache subdir. Reopened regression of NousResearch#34485/NousResearch#38108, neither of which covered the profile-scoped symlink case. Fixes NousResearch#31733.
Root-run gateways have $HOME=/root, which is on the MEDIA system-path denylist, so the gateway silently dropped agent-generated deliverables under /root (e.g. /root/work/proposal.docx) — the user got a 'here is your file' reply with nothing attached. _path_under_denied_prefix now treats the running user's own home as deliverable: the home tree itself is no longer denied, while the more-specific denied paths inside it (~/.ssh, ~/.aws, ~/.hermes/.env, auth.json, config.yaml) stay blocked because they are separate denylist entries. The exception only matches when the denied prefix IS $HOME, so a non-root gateway still can't deliver another user's home. Diagnosis, reproduction, and the failing-case analysis are from @GodsBoy (NousResearch#38108 / NousResearch#38106). Implemented here as the minimal denylist fix rather than a staging/copy subsystem. Co-authored-by: GodsBoy <dhuysamen@gmail.com>
Root-run gateways have $HOME=/root, which is on the MEDIA system-path denylist, so the gateway silently dropped agent-generated deliverables under /root (e.g. /root/work/proposal.docx) — the user got a 'here is your file' reply with nothing attached. _path_under_denied_prefix now treats the running user's own home as deliverable: the home tree itself is no longer denied, while the more-specific denied paths inside it (~/.ssh, ~/.aws, ~/.hermes/.env, auth.json, config.yaml) stay blocked because they are separate denylist entries. The exception only matches when the denied prefix IS $HOME, so a non-root gateway still can't deliver another user's home. Diagnosis, reproduction, and the failing-case analysis are from @GodsBoy (NousResearch#38108 / NousResearch#38106). Implemented here as the minimal denylist fix rather than a staging/copy subsystem. Co-authored-by: GodsBoy <dhuysamen@gmail.com>
…HOME (NousResearch#54060) Generated images under a profile gateway's cache (profiles/<name>/cache/ images/...) were silently dropped from Telegram/Discord delivery when HERMES_HOME is symlinked under a denied prefix (e.g. /opt/data -> /root/.hermes) and $HOME is not that prefix. The resolved path lands under /root (a system denylist prefix), the root-home exception only fires when the denied prefix IS $HOME, and the static safe-roots list only covers the active HERMES_HOME's top-level cache — not per-profile cache dirs. Both gates fail, so validate_media_delivery_path returns None and the gateway logs 'Skipping unsafe MEDIA directive path'. _media_delivery_allowed_roots() now also enumerates per-profile cache roots (<root>/profiles/*/cache/{images,audio,videos,documents, screenshots}) at check time. Allowlist match runs before the denylist, so the profile artifact delivers regardless of the /root interaction; profile-dir credentials (auth.json) stay blocked since they aren't under a cache subdir. Reopened regression of NousResearch#34485/NousResearch#38108, neither of which covered the profile-scoped symlink case. Fixes NousResearch#31733.
Root-run gateways have $HOME=/root, which is on the MEDIA system-path denylist, so the gateway silently dropped agent-generated deliverables under /root (e.g. /root/work/proposal.docx) — the user got a 'here is your file' reply with nothing attached. _path_under_denied_prefix now treats the running user's own home as deliverable: the home tree itself is no longer denied, while the more-specific denied paths inside it (~/.ssh, ~/.aws, ~/.hermes/.env, auth.json, config.yaml) stay blocked because they are separate denylist entries. The exception only matches when the denied prefix IS $HOME, so a non-root gateway still can't deliver another user's home. Diagnosis, reproduction, and the failing-case analysis are from @GodsBoy (NousResearch#38108 / NousResearch#38106). Implemented here as the minimal denylist fix rather than a staging/copy subsystem. Co-authored-by: GodsBoy <dhuysamen@gmail.com>
Root-run gateways have $HOME=/root, which is on the MEDIA system-path denylist, so the gateway silently dropped agent-generated deliverables under /root (e.g. /root/work/proposal.docx) — the user got a 'here is your file' reply with nothing attached. _path_under_denied_prefix now treats the running user's own home as deliverable: the home tree itself is no longer denied, while the more-specific denied paths inside it (~/.ssh, ~/.aws, ~/.hermes/.env, auth.json, config.yaml) stay blocked because they are separate denylist entries. The exception only matches when the denied prefix IS $HOME, so a non-root gateway still can't deliver another user's home. Diagnosis, reproduction, and the failing-case analysis are from @GodsBoy (NousResearch#38108 / NousResearch#38106). Implemented here as the minimal denylist fix rather than a staging/copy subsystem. Co-authored-by: GodsBoy <dhuysamen@gmail.com>
…HOME (NousResearch#54060) Generated images under a profile gateway's cache (profiles/<name>/cache/ images/...) were silently dropped from Telegram/Discord delivery when HERMES_HOME is symlinked under a denied prefix (e.g. /opt/data -> /root/.hermes) and $HOME is not that prefix. The resolved path lands under /root (a system denylist prefix), the root-home exception only fires when the denied prefix IS $HOME, and the static safe-roots list only covers the active HERMES_HOME's top-level cache — not per-profile cache dirs. Both gates fail, so validate_media_delivery_path returns None and the gateway logs 'Skipping unsafe MEDIA directive path'. _media_delivery_allowed_roots() now also enumerates per-profile cache roots (<root>/profiles/*/cache/{images,audio,videos,documents, screenshots}) at check time. Allowlist match runs before the denylist, so the profile artifact delivers regardless of the /root interaction; profile-dir credentials (auth.json) stay blocked since they aren't under a cache subdir. Reopened regression of NousResearch#34485/NousResearch#38108, neither of which covered the profile-scoped symlink case. Fixes NousResearch#31733.
What does this PR do?
Agents routinely produce deliverables in a working directory (
proposal.docx,report.pdf) instead of inside a Hermes cache. When the gateway runs as root, the whole/roothome is on the MEDIA delivery denylist, sovalidate_media_delivery_pathrejects the path and the attachment is silently dropped while the chat reply still claims a file is attached.Live repro: a
MEDIA:/root/clawd/...docxdirective producedSkipping unsafe MEDIA directive path: /root/clawd/...docxingateway.log, and the Telegram DM arrived with no attachment. Copying the same file to~/.hermes/cache/documents/...docxand sending that cache path delivered fine.This adds
stage_media_delivery_path: when a path fails direct validation, a safe local deliverable is copied into the Hermes media cache that matches its extension, and the staged path (now under an allowlisted cache root) is delivered normally.filter_media_delivery_pathsandfilter_local_delivery_pathstry direct validation first, then the staging fallback, then rejection, so every dispatch site benefits.The denylist is preserved, and no arbitrary working directory is allowlisted:
_media_delivery_denied_paths(/etc,~/.ssh,~/.hermes/.env, ...)..netrc,id_rsa,*.pem,auth.json,config.yaml, ...)./root/work/proposal.docxwhile/root/.sshstays blocked. It cannot un-block a credential sub-directory or another user's home.This preserves the denylist protections and avoids allowlisting
Path.home(),/root,/tmp, or arbitrary working dirs.Related Issue
Fixes #38106
Type of Change
Changes Made
gateway/platforms/base.py_resolve_media_candidatefromvalidate_media_delivery_path(shared, behaviour-preserving resolution prologue)._resolved_under_allowed_root,_stage_filename_is_credential,_stage_path_under_denied_dir,_stage_target_cache_dir,_copy_into_media_cache, andstage_media_delivery_path.BasePlatformAdapter.filter_media_delivery_pathsandfilter_local_delivery_paths, and expose astage_media_delivery_pathstatic method.import shutil.tests/gateway/test_platform_base.pyTestMediaDeliveryStaging(21 tests).How to Test
Manual (root-run gateway, default non-strict mode):
/root/work/proposal.docx.BasePlatformAdapter.validate_media_delivery_path("/root/work/proposal.docx")returnsNone(denied prefix).BasePlatformAdapter.filter_media_delivery_paths([("/root/work/proposal.docx", False)])returns the staged~/.hermes/cache/documents/proposal.docx, which then validates and delivers. The source file is untouched.~/.ssh/id_rsa,~/.hermes/.env,/etc/..., and a.netrc/*.pemin the workdir all remain rejected (not staged).Checklist
Code
fix(gateway):)Documentation & Housekeeping
cli-config.yaml.exampleif I added/changed config keys — N/A (no new config keys)CONTRIBUTING.mdorAGENTS.mdif I changed architecture or workflows — N/Apathlib/shutil, home-relative denylist resolution; macOS Keychains already deniedScreenshots / Logs
Before (gateway.log):
Skipping unsafe MEDIA directive path: /root/clawd/...docxand no attachment.After:
Staged MEDIA file /root/.../...docx -> ~/.hermes/.../...docx for deliveryand the file attaches.