Skip to content

feat: add file/image attachment support to WASM channels - #409

Closed
sergeytimoshin wants to merge 4 commits into
nearai:mainfrom
sergeytimoshin:feat/attachment-support
Closed

sergeytimoshin wants to merge 4 commits into
nearai:mainfrom
sergeytimoshin:feat/attachment-support

Conversation

@sergeytimoshin

Copy link
Copy Markdown

Enable the agent to download files via the http tool and send them as attachments through WASM channels (photos on Telegram, documents elsewhere).

Changes:

  • WIT: add attachment record, extend agent-response, add on-broadcast export
  • WASM wrapper: wire attachments through respond/broadcast, read files from disk, detect MIME types, enforce 50MB size cap
  • Telegram: implement multipart file upload (sendPhoto/sendDocument), support attachments in on_respond and on_broadcast, add polling_enabled config flag, reduce poll timeout to 25s to avoid WASM callback race
  • HTTP tool: add save_to parameter for binary file downloads to /tmp, allow redirects (up to 10 hops) instead of blocking 3xx responses
  • Message tool: allow attachment paths from /tmp in addition to sandbox dir, update description to document the download-then-attach workflow
  • Credentials: fall back to environment variables when secrets store has no matching entry for a channel credential
  • Discord/Slack/WhatsApp: add stub on_broadcast implementations

Enable the agent to download files via the http tool and send them as
attachments through WASM channels (photos on Telegram, documents
elsewhere).

Changes:
- WIT: add `attachment` record, extend `agent-response`, add `on-broadcast` export
- WASM wrapper: wire attachments through respond/broadcast, read files
  from disk, detect MIME types, enforce 50MB size cap
- Telegram: implement multipart file upload (sendPhoto/sendDocument),
  support attachments in on_respond and on_broadcast, add polling_enabled
  config flag, reduce poll timeout to 25s to avoid WASM callback race
- HTTP tool: add `save_to` parameter for binary file downloads to /tmp,
  allow redirects (up to 10 hops) instead of blocking 3xx responses
- Message tool: allow attachment paths from /tmp in addition to sandbox
  dir, update description to document the download-then-attach workflow
- Credentials: fall back to environment variables when secrets store
  has no matching entry for a channel credential
- Discord/Slack/WhatsApp: add stub on_broadcast implementations

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings February 28, 2026 01:44
@github-actions github-actions Bot added scope: channel/wasm WASM channel runtime scope: tool/builtin Built-in tools scope: dependencies Dependency updates size: XL 500+ changed lines risk: medium Business logic, config, or moderate-risk modules contributor: new First-time contributor labels Feb 28, 2026
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly expands the agent's communication capabilities by integrating file and image attachment support across WASM channels. It enables agents to download files using the HTTP tool and subsequently send them as attachments through various channels, with specific implementations for Telegram. The changes also improve credential management by allowing environment variable fallbacks and refine HTTP request handling.

Highlights

  • File and Image Attachment Support: Introduced comprehensive file and image attachment support for WASM channels, allowing agents to send various media types through on_respond and the new on_broadcast functions.
  • Telegram Channel Enhancements: Enhanced the Telegram channel to handle multipart file uploads (photos and documents), support attachments in on_respond and on_broadcast, added a polling_enabled configuration option, and reduced the poll timeout to 25 seconds.
  • HTTP Tool Updates: Updated the HTTP tool with a save_to parameter for downloading binary files to /tmp and enabled automatic redirection following for up to 10 hops, removing the previous redirect blocking.
  • Message Tool Flexibility: Modified the Message tool to permit attachment paths from the /tmp directory, alongside existing sandbox paths, facilitating a download-then-attach workflow.
  • Credential Management Improvement: Implemented a fallback mechanism for channel credentials, allowing them to be sourced from environment variables if not found in the secrets store.
  • WASM Channel Interface Extension: Extended the WASM channel interface (wit/channel.wit) with an attachment record and an on-broadcast export, and wired these through the WASM wrapper to enable proactive messages with attachments.
Changelog
  • Cargo.lock
    • Added console package.
    • Updated syn dependency to version 2.0.117.
  • channels-src/discord/src/lib.rs
    • Added a stub implementation for the on_broadcast function.
  • channels-src/slack/src/lib.rs
    • Added a stub implementation for the on_broadcast function.
  • channels-src/telegram/src/lib.rs
    • Imported the Attachment type from the channel WIT interface.
    • Introduced a polling_enabled flag to the TelegramConfig struct.
    • Adjusted the webhook mode logic to prioritize polling_enabled over tunnel_url presence.
    • Reduced the get_updates_url poll timeout from 30s to 25s.
    • Modified on_respond to send attachments before text content.
    • Implemented the on_broadcast function to send messages with attachments.
    • Added helper functions write_multipart_field, write_multipart_file, send_photo, send_document, and send_attachment for handling multipart file uploads to Telegram.
  • channels-src/whatsapp/src/lib.rs
    • Added a stub implementation for the on_broadcast function.
  • src/channels/wasm/wrapper.rs
    • Updated the call_on_respond function signature to include an attachments parameter.
    • Modified call_on_respond to read attachment files from disk before passing them to WASM.
    • Implemented the call_on_broadcast function, including reading attachments from disk.
    • Updated the Channel trait implementation for send and broadcast to pass attachment paths.
    • Added mime_from_extension to determine MIME types from file extensions.
    • Defined MAX_TOTAL_ATTACHMENT_BYTES for attachment size limits.
    • Included a test for mime_from_extension.
  • src/main.rs
    • Updated inject_channel_credentials to fall back to environment variables for credentials not found in the secrets store.
  • src/tools/builtin/http.rs
    • Configured the HTTP client to follow up to 10 redirects.
    • Updated the tool description to explain the save_to parameter for binary downloads.
    • Added save_to to the parameters schema, requiring paths under /tmp/.
    • Removed the explicit blocking of 3xx redirects.
    • Implemented logic to save the HTTP response body to a specified file path.
  • src/tools/builtin/message.rs
    • Updated the tool description to document the download-then-attach workflow for attachments.
    • Modified attachment path validation to allow files located in /tmp/ in addition to the sandbox directory.
    • Adjusted a test case to reflect the updated attachment path validation rules.
  • wit/channel.wit
    • Defined a new attachment record with filename, mime-type, and data fields.
    • Extended the agent-response record to include a list of attachment records.
    • Added a new on-broadcast function export to the channel interface.
Activity
  • No human activity has been recorded on this pull request yet.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces file and image attachment support for WASM channels. However, critical security vulnerabilities have been identified, including a path traversal vulnerability in the HttpTool allowing arbitrary file writes, SSRF attacks due to automatic redirect following without re-validation, and a multipart header injection flaw in the Telegram channel's attachment handling. Additionally, while several code duplication issues were noted, these refactorings are suggested to be deferred to follow-up tasks to maintain focus on the current feature implementation. A suggestion to improve MIME type detection by utilizing the mime_guess crate is also included.

I am having trouble creating individual review comments. Click here to see my feedback.

src/tools/builtin/http.rs (410-416)

security-critical critical

The HttpTool introduces a save_to parameter that allows saving the response body to a file. The validation check !save_to.starts_with("/tmp/") is insufficient to prevent path traversal attacks. An attacker (e.g., via prompt injection) can provide a path like /tmp/../../etc/passwd which starts with /tmp/ but resolves to a location outside the intended directory. This allows overwriting arbitrary files on the host system with content fetched from a URL.

src/tools/builtin/http.rs (40)

security-high high

The HttpTool was modified to follow redirects automatically (up to 10 hops). While the initial URL is validated against a blocklist of private IPs and localhost in validate_url, subsequent URLs in a redirect chain are not re-validated. An attacker can provide a URL that points to a malicious server, which then redirects the client to a sensitive internal resource (e.g., https://127.0.0.1/admin or cloud metadata services), bypassing the SSRF protections.

channels-src/telegram/src/lib.rs (892-896)

security-medium medium

The write_multipart_file function constructs a Content-Disposition header by directly embedding the filename without sanitization or escaping. The filename is derived from the file path provided in the tool parameters. If a filename contains double quotes or newline characters, an attacker can inject additional parameters or headers into the multipart request.

channels-src/telegram/src/lib.rs (654-679)

medium

The logic in on_broadcast for sending attachments and then the message content is very similar to the logic in on_respond. This duplication can be refactored into a shared helper function to improve maintainability. However, for this feature-focused PR, consider tracking this refactoring as a follow-up task.

References
  1. Consolidate related sequences of operations, such as creating, persisting, and scheduling a job, into a single reusable method to improve code consistency and maintainability.
  2. Defer large structural refactorings, like breaking down a large file, when the primary goal of a pull request is to achieve correctness and feature parity with an existing implementation. The refactoring should be tracked as a follow-up task.

channels-src/telegram/src/lib.rs (906-1013)

medium

The functions send_photo and send_document are nearly identical. This duplicated code can be extracted into a single helper function to reduce redundancy and improve maintainability. For this feature-focused PR, consider tracking this refactoring as a follow-up task.

References
  1. Consolidate related sequences of operations, such as creating, persisting, and scheduling a job, into a single reusable method to improve code consistency and maintainability.
  2. Defer large structural refactorings, like breaking down a large file, when the primary goal of a pull request is to achieve correctness and feature parity with an existing implementation. The refactoring should be tracked as a follow-up task.

src/channels/wasm/wrapper.rs (1254-1363)

medium

This new call_on_broadcast function is almost a complete copy of call_on_respond, introducing significant duplicated code. While refactoring this common logic into a generic execute_wasm_callback helper function would drastically reduce duplication and improve maintainability, for this feature-focused PR, it is recommended to track this as a follow-up task.

References
  1. Consolidate related sequences of operations, such as creating, persisting, and scheduling a job, into a single reusable method to improve code consistency and maintainability.
  2. Defer large structural refactorings, like breaking down a large file, when the primary goal of a pull request is to achieve correctness and feature parity with an existing implementation. The refactoring should be tracked as a follow-up task.

src/channels/wasm/wrapper.rs (2517-2555)

medium

This manual implementation for detecting MIME types from file extensions is quite extensive and could be hard to maintain. The mime_guess crate is already a dependency in the project and provides a more robust and comprehensive way to handle this.

Using mime_guess would simplify this code and ensure you're using a standard, well-maintained library for MIME detection.

fn mime_from_extension(path: &str) -> String {
    mime_guess::from_path(path)
        .first_or_octet_stream()
        .to_string()
}

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds end-to-end support for downloading files and sending them as attachments through WASM-backed channels (with Telegram implementing real uploads and other channels adding scaffolding).

Changes:

  • Extend channel WIT types to include attachment records on agent-response and add an on-broadcast export for proactive sends.
  • Wire attachment paths through the Rust host WASM wrapper (read files, infer MIME types, enforce a total size cap) and expose broadcast behavior.
  • Update built-in tools and channels: HTTP tool adds save_to downloads and redirect following; Message tool allows /tmp attachments; Telegram adds multipart uploads; other WASM channels stub on_broadcast.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 10 comments.

Show a summary per file
File Description
wit/channel.wit Adds attachment type + attachments on responses + on-broadcast callback definition.
src/tools/builtin/message.rs Documents download→attach flow; allows attachment paths from /tmp as well as base dir; updates tests.
src/tools/builtin/http.rs Enables redirects and adds save_to to write response bytes to disk.
src/main.rs Adds env-var fallback when injecting channel credentials.
src/channels/wasm/wrapper.rs Reads attachment files into WIT attachments, adds call_on_broadcast and Channel broadcast, adds MIME helper.
channels-src/telegram/src/lib.rs Supports attachments by multipart upload via sendPhoto/sendDocument; adds polling_enabled; adjusts poll timeout.
channels-src/whatsapp/src/lib.rs Adds stub on_broadcast.
channels-src/slack/src/lib.rs Adds stub on_broadcast.
channels-src/discord/src/lib.rs Adds stub on_broadcast.
Cargo.lock Updates dependency lockfile to include new/updated crates.
Comments suppressed due to low confidence (3)

src/channels/wasm/wrapper.rs:2578

  • read_attachments reads each entire file into memory (std::fs::read) before enforcing the 50MB total size cap. A single oversized file can cause large allocations / OOM before you ever hit the limit check. Consider pre-checking sizes via metadata().len() (and/or streaming reads with an upper bound) before loading bytes into memory.
            extension_name,
            success,
            message,
        } => wit_channel::StatusUpdate {
            status: wit_channel::StatusType::AuthCompleted,
            message: format!(
                "Authentication {} for {}. {}",
                if *success { "completed" } else { "failed" },
                extension_name,
                message
            ),

src/channels/wasm/wrapper.rs:2571

  • read_attachments reads arbitrary host filesystem paths provided via OutgoingResponse.attachments without any sandbox/allowlist validation. This means any caller that can influence attachments could exfiltrate local files through the WASM channel. Consider validating attachment paths here as a defense-in-depth measure (e.g., restrict to ironclaw_base_dir() and /tmp using path_utils::validate_path).
            extension_name,
            success,
            message,
        } => wit_channel::StatusUpdate {

src/tools/builtin/http.rs:370

  • The comment about still seeing a 3xx status meaning the redirect chain was too long is inaccurate with Policy::limited(...) (reqwest errors on too many redirects rather than returning a final 3xx). Also 3xx statuses like 304 aren’t redirects. Either explicitly handle 3xx responses here (with a clear error) or adjust the comment to reflect actual behavior.
        let status = response.status().as_u16();

        // Redirects are followed automatically (up to 10 hops).
        // If we still see a 3xx here, the chain was too long.

        let headers: HashMap<String, String> = response
            .headers()
            .iter()
            .filter_map(|(k, v)| v.to_str().ok().map(|v| (k.to_string(), v.to_string())))
            .collect();

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread channels-src/whatsapp/src/lib.rs
Comment thread channels-src/slack/src/lib.rs
Comment thread src/tools/builtin/http.rs Outdated
Comment thread channels-src/telegram/src/lib.rs
Comment thread channels-src/discord/src/lib.rs
Comment thread src/tools/builtin/http.rs Outdated
Comment thread src/tools/builtin/http.rs Outdated
Comment thread src/tools/builtin/http.rs
Comment thread src/tools/builtin/http.rs
Comment thread channels-src/telegram/src/lib.rs
Copilot AI review requested due to automatic review settings February 28, 2026 01:56
- Fix path traversal in http save_to: use path_utils::validate_path
  with /tmp as base dir to catch traversal and symlink escapes
- Fix SSRF via redirect chain: custom redirect policy validates each
  hop against IP/host blocklist AND rejects https→http scheme downgrades
- Fix multipart filename injection in Telegram: sanitize filenames by
  stripping quotes, newlines, backslashes, and non-ASCII characters
- Replace manual mime_from_extension with mime_guess crate
- Use spawn_blocking for filesystem ops in save_to (avoid blocking
  the Tokio runtime)
- Increase save_to size limit to 50MB (matches attachment cap) while
  keeping text responses at 5MB
- Restrict Telegram sendPhoto to known-safe formats (jpeg/png/gif/webp),
  send all other image types as documents
- Remove duplicate broadcast implementations in wrapper.rs
- Add unit tests for save_to path validation (traversal, prefix, success)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@sergeytimoshin
sergeytimoshin force-pushed the feat/attachment-support branch from bacad06 to eb13580 Compare February 28, 2026 02:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (4)

src/channels/wasm/wrapper.rs:2676

  • read_attachments accepts arbitrary filesystem paths and calls std::fs::read directly. Since attachments can be set by the agent/tooling, this creates a local file read primitive unless paths are validated at a single choke point. Consider validating each path with the same sandbox rules as MessageTool (e.g., only under ~/.ironclaw and/or a dedicated /tmp/ironclaw download dir) before reading.
    use crate::channels::wasm::runtime::{
        PreparedChannelModule, WasmChannelRuntime, WasmChannelRuntimeConfig,
    };
    use crate::channels::wasm::wrapper::{HttpResponse, WasmChannel};

src/channels/wasm/wrapper.rs:2278

  • impl Channel for WasmChannel now contains two broadcast methods (one earlier that ignores user_id and calls call_on_respond, and this new one calling call_on_broadcast). This will not compile and also leaves a stale broadcast path that doesn't pass the new attachments arg to call_on_respond.
    }

    async fn shutdown(&self) -> Result<(), ChannelError> {
        // Cancel typing indicator
        self.cancel_typing_task().await;

        // Send shutdown signal
        if let Some(tx) = self.shutdown_tx.write().await.take() {
            let _ = tx.send(());
        }

        // Stop polling by dropping the sender (receiver will complete)

src/channels/wasm/wrapper.rs:2403

  • impl Channel for SharedWasmChannel defines broadcast twice, which will not compile. Keep only one implementation that forwards to inner.broadcast(...).
                },
            )
            .collect(),
        poll: wit
            .poll
            .map(|p| crate::channels::wasm::schema::PollConfigSchema {
                interval_ms: p.interval_ms,

src/channels/wasm/wrapper.rs:2683

  • read_attachments reads the entire file into memory before enforcing MAX_TOTAL_ATTACHMENT_BYTES, so a single large file can temporarily allocate far beyond the limit (and potentially OOM) even though it later errors. Consider checking std::fs::metadata(path)?.len() (and running total) before reading, or reading with an upper bound.
    use crate::channels::wasm::runtime::{
        PreparedChannelModule, WasmChannelRuntime, WasmChannelRuntimeConfig,
    };
    use crate::channels::wasm::wrapper::{HttpResponse, WasmChannel};
    use crate::pairing::PairingStore;
    use crate::tools::wasm::ResourceLimits;

    fn create_test_channel() -> WasmChannel {
        let config = WasmChannelRuntimeConfig::for_testing();
        let runtime = Arc::new(WasmChannelRuntime::new(config).unwrap());


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/tools/builtin/http.rs
Comment thread src/tools/builtin/http.rs
Comment thread src/tools/builtin/message.rs
Verifies that the message tool accepts attachment paths under /tmp/
(path validation passes, fails at channel send since no real channel
is connected). Prevents regressions on the /tmp allowlist.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

@zmanian zmanian left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good feature idea -- attachment support through WASM channels is needed. The Telegram multipart implementation and WIT interface extensions show solid understanding of the system. However, there are several security issues that need to be addressed before merge.

Critical

1. create_dir_all before path validation (TOCTOU in validate_save_to_path).
In src/tools/builtin/http.rs, create_dir_all(parent) is called before validate_path. A path like /tmp/../../var/evil/ passes the starts_with("/tmp/") prefix check, then create_dir_all creates directories outside /tmp before validate_path rejects it. Move create_dir_all after validation succeeds.

2. WIT breaking changes with no backward compatibility strategy.
Adding attachments to agent-response and the new on-broadcast export are breaking changes. Pre-compiled WASM channels that don't export on-broadcast will fail to instantiate. Consider making on-broadcast optional in the WIT world, or detect its absence and fall back gracefully. At minimum, document this as a breaking WIT change.

High

3. SSRF redirect policy fails open on DNS resolution failure.
In the custom redirect policy in http.rs, when to_socket_addrs() returns Err, the redirect is allowed (attempt.follow()). Fail-closed: reject redirects when DNS resolution fails.

4. read_attachments has no path validation (defense-in-depth).
In src/channels/wasm/wrapper.rs, read_attachments reads arbitrary file paths with no validation. While the message tool validates paths upstream, defense-in-depth requires read_attachments to independently validate paths are within allowed directories (/tmp or sandbox base dir).

5. Credential env-var fallback should be a separate PR.
The inject_channel_credentials change in main.rs bypasses the encrypted secrets store for env var fallback. This is orthogonal to attachments and has security implications (credentials visible in /proc/<pid>/environ). Split this out.

Medium

6. Stub on_broadcast returns Ok(()) on silent message drop.
Discord/Slack/WhatsApp stubs return Ok(()), telling the caller delivery succeeded when the message was actually dropped. Return Err("Broadcast not implemented for this channel") instead.

7. Predictable multipart boundary.
format!("ironclaw-{}", channel_host::now_millis()) is predictable. If the boundary appears in attachment data, the multipart message will be malformed. Use a random or hash-based boundary.

8. Unrelated polling_enabled change bundled in.
The polling_enabled config flag and poll timeout change (30s to 25s) in the Telegram channel are unrelated to attachments. Separate commit or PR.

Low

  • content_type not sanitized in multipart headers (header injection if \r\n present)
  • 50 MB fully buffered in memory before disk write -- consider streaming for large files
  • Code duplication between Telegram on_respond and on_broadcast for attachment sending

Nice first contribution -- the overall architecture is right. Please fix the security items and split the credential fallback into its own PR.

ilblackdragon added a commit that referenced this pull request Mar 6, 2026
…409)

Reconcile PR #409's outbound attachment work with our inbound attachment
support into a unified design:

WIT type split:
- `inbound-attachment` in channel-host: metadata-only (id, mime_type,
  filename, size_bytes, source_url, storage_key, extracted_text)
- `attachment` in channel: raw bytes (filename, mime_type, data) on
  agent-response for outbound sending

Outbound features (from PR #409):
- `on-broadcast` WIT export for proactive messages without prior inbound
- Telegram: multipart sendPhoto/sendDocument with auto photo→document
  fallback for files >10MB
- wrapper.rs: `call_on_broadcast`, `read_attachments` from disk,
  attachment params threaded through `call_on_respond`
- HTTP tool: `save_to` param for binary downloads to /tmp/ (50MB limit,
  path traversal protection, SSRF-safe redirect following)
- Message tool: allow /tmp/ paths for attachments alongside base_dir
- Credential env var fallback in inject_channel_credentials

Channel updates:
- All 4 channels implement on_broadcast (Telegram full, others stub)
- Telegram: polling_enabled config, adjusted poll timeout
- Inbound attachment types renamed to InboundAttachment in all channels

Tests: 1965 passing (9 new), 0 clippy warnings

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ilblackdragon added a commit that referenced this pull request Mar 7, 2026
* feat: add inbound attachment support to WASM channel system

Add attachment record to WIT interface and implement inbound media
parsing across all four channel implementations (Telegram, Slack,
WhatsApp, Discord). Attachments flow from WASM channels through
EmittedMessage to IncomingMessage with validation (size limits,
MIME allowlist, count caps) at the host boundary.

- Add `attachment` record to `emitted-message` in wit/channel.wit
- Add `IncomingAttachment` struct to channel.rs and re-export
- Add host-side validation (20MB total, 10 max, MIME allowlist)
- Telegram: parse photo, document, audio, video, voice, sticker
- Slack: parse file attachments with url_private
- WhatsApp: parse image, audio, video, document with captions
- Discord: backward-compatible empty attachments
- Update FEATURE_PARITY.md section 7
- Add fixture-based tests per channel and host integration tests

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: integrate outbound attachment support and reconcile WIT types (#409)

Reconcile PR #409's outbound attachment work with our inbound attachment
support into a unified design:

WIT type split:
- `inbound-attachment` in channel-host: metadata-only (id, mime_type,
  filename, size_bytes, source_url, storage_key, extracted_text)
- `attachment` in channel: raw bytes (filename, mime_type, data) on
  agent-response for outbound sending

Outbound features (from PR #409):
- `on-broadcast` WIT export for proactive messages without prior inbound
- Telegram: multipart sendPhoto/sendDocument with auto photo→document
  fallback for files >10MB
- wrapper.rs: `call_on_broadcast`, `read_attachments` from disk,
  attachment params threaded through `call_on_respond`
- HTTP tool: `save_to` param for binary downloads to /tmp/ (50MB limit,
  path traversal protection, SSRF-safe redirect following)
- Message tool: allow /tmp/ paths for attachments alongside base_dir
- Credential env var fallback in inject_channel_credentials

Channel updates:
- All 4 channels implement on_broadcast (Telegram full, others stub)
- Telegram: polling_enabled config, adjusted poll timeout
- Inbound attachment types renamed to InboundAttachment in all channels

Tests: 1965 passing (9 new), 0 clippy warnings

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add audio transcription pipeline and extensible WIT attachment design

Add host-side transcription middleware (OpenAI Whisper) that detects audio
attachments with inline data on incoming messages and transcribes them
automatically. Refactor WIT inbound-attachment to use extras-json and a
store-attachment-data host function instead of typed fields, so future
attachment properties (dimensions, codec, etc.) don't require WIT changes
that invalidate all channel plugins.

- Add src/transcription/ module: TranscriptionProvider trait,
  TranscriptionMiddleware, AudioFormat enum, OpenAI Whisper provider
- Add src/config/transcription.rs: TRANSCRIPTION_ENABLED/MODEL/BASE_URL
- Wire middleware into agent message loop via AgentDeps
- WIT: replace data + duration-secs with extras-json + store-attachment-data
- Host: parse extras-json for well-known keys, merge stored binary data
- Telegram: download voice files via store-attachment-data, add duration
  to extras-json, add /file/bot to HTTP allowlist, voice-only placeholder
- Add reqwest multipart feature for Whisper API uploads
- 5 regression tests for transcription middleware

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: wire attachment processing into LLM pipeline with multimodal image support

Attachments on incoming messages are now augmented into user text via XML tags
before entering the turn system, and images with data are passed as multimodal
content parts (base64 data URIs) to LLM providers. This enables audio transcripts,
document text, and image content to reach the LLM without changes to ChatMessage
serialization or provider interfaces.

- Add src/agent/attachments.rs with augment_with_attachments() and 9 unit tests
- Add ContentPart/ImageUrl types to llm::provider with OpenAI-compatible serde
- Carry image_content_parts transiently on Turn (skipped in serialization)
- Update nearai_chat and rig_adapter to serialize multimodal content
- Add 3 e2e tests verifying attachments flow through the full agent loop

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: CI failures — formatting, version bumps, and Telegram voice test

- Fix cargo fmt formatting in attachments.rs, nearai_chat.rs, rig_adapter.rs,
  e2e_attachments.rs
- Bump channel registry versions 0.1.0 → 0.2.0 (discord, slack, telegram,
  whatsapp) to satisfy version-bump CI check
- Fix Telegram test_extract_attachments_voice: add missing required `duration`
  field to voice fixture JSON

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: bump WIT channel version to 0.3.0, fix Telegram voice test, add pre-commit hook

- Bump wit/channel.wit package version 0.2.0 → 0.3.0 (interface changed with
  store-attachment-data)
- Update WIT_CHANNEL_VERSION constant and registry wit_version fields to match
- Fix Telegram test_extract_attachments_voice: gate voice download behind
  #[cfg(target_arch = "wasm32")] so host functions aren't called in native tests,
  update assertions for generated filename and extras_json duration
- Add @0.3.0 linker stubs in wit_compat.rs
- Add .githooks/pre-commit hook that runs scripts/check-version-bumps.sh when
  WIT or extension sources are staged
- Symlink commit-msg regression hook into .githooks/

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: extract voice download from extract_attachments into handle_message

Move download_voice_file + store_attachment_data calls out of
extract_attachments into a separate download_and_store_voice function
called from handle_message. This keeps extract_attachments as a pure
data-mapping function with no host calls, making it fully testable
in native unit tests without #[cfg(target_arch)] gates.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review comments — security, correctness, and code quality

Security fixes:
- Add path validation to read_attachments (restrict to /tmp/) preventing
  arbitrary file reads from compromised tools
- Escape XML special characters in attachment filenames, MIME types, and
  extracted text to prevent prompt injection via tag spoofing
- Percent-encode file_id in Telegram getFile URL to prevent query injection
- Clone SecretString directly instead of expose_secret().to_string()

Correctness fixes:
- Fix store_attachment_data overwrite accounting: subtract old entry size
  before adding new to prevent inflated totals and false rejections
- Use max(reported, stored_size) for attachment size accounting to prevent
  WASM channels from under-reporting size_bytes to bypass limits
- Add application/octet-stream to MIME allowlist (channels default unknown
  types to this)

Code quality:
- Extract send_response helper in Telegram, deduplicating on_respond and
  on_broadcast
- Rename misleading Discord test to test_parse_slash_command_interaction
- Fix .githooks/commit-msg to use relative symlink (portable across machines)

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add tool_upgrade command + fix TOCTOU in save_to path validation

Add `tool_upgrade` — a new extension management tool that automatically
detects and reinstalls WASM extensions with outdated WIT versions.
Preserves authentication secrets during upgrade. Supports upgrading a
single extension by name or all installed WASM tools/channels at once.

Fix TOCTOU in `validate_save_to_path`: validate the path *before*
creating parent directories, so traversal paths like `/tmp/../../etc/`
cannot cause filesystem mutations outside /tmp before being rejected.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: unify WIT package version to 0.3.0 across tool.wit and all capabilities

tool.wit and channel.wit share the `near:agent` package namespace, so they
must declare the same version. Bumps tool.wit from 0.2.0 to 0.3.0 and
updates all capabilities files and registry entries to match.

Fixes `cargo component build` failure: "package identifier near:agent@0.2.0
does not match previous package name of near:agent@0.3.0"

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: move WIT file comments after package declaration

WIT treats `//` comments before `package` as doc comments. When both
tool.wit and channel.wit had header comments, the parser rejected them
as "doc comments on multiple 'package' items". Move comments after the
package declaration in both files.

Also bumps tool registry versions to 0.2.0 to match the WIT 0.3.0 bump.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: display extension versions in gateway Extensions tab

Add version field to InstalledExtension and RegistryEntry types, pipe
through the web API (ExtensionInfo, RegistryEntryInfo), and render as
a badge in the gateway UI for both installed and available extensions.

For installed WASM extensions, version is read from the capabilities
file with a fallback to the registry entry when the local file has no
version (old installations). Bump all extension Cargo.toml and registry
JSON versions from 0.1.0 to 0.2.0 to keep them in sync.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add document text extraction middleware for PDF, Office, and text files

Extract text from document attachments (PDF, DOCX, PPTX, XLSX, RTF, plain text,
code files) so the LLM can reason about uploaded documents. Uses pdf-extract for
PDFs, zip+XML parsing for Office XML formats, and UTF-8 decode for text files.
Wired into the agent loop after transcription middleware.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: download document files in Telegram channel for text extraction

The DocumentExtractionMiddleware needs file bytes in the attachment `data`
field, but only voice files were being downloaded. Document attachments
(PDFs, DOCX, etc.) had empty `data` and a source_url with a credential
placeholder that only works inside the WASM host's http_request.

Add `download_and_store_documents()` that downloads non-voice, non-image,
non-audio attachments via the existing two-step getFile→download flow and
stores bytes via `store_attachment_data` for host-side extraction.

Also rename `download_voice_file` → `download_telegram_file` since it's
generic for any file_id.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: allow Office MIME types and increase file download limit for Telegram

Two issues preventing document extraction from Telegram:

1. PPTX/DOCX/XLSX MIME types (application/vnd.*) were dropped by the
   WASM host attachment allowlist — add application/vnd., application/msword,
   and application/rtf prefixes.

2. Telegram file downloads over 10 MB failed with "Response body too large" —
   set max_response_bytes to 20 MB in Telegram capabilities.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: report document extraction errors back to user instead of silently skipping

- Bump max_response_bytes to 50 MB for Telegram file downloads
- When document extraction fails (too large, download error, parse error),
  set extracted_text to a user-friendly error message instead of leaving it
  None. This ensures the LLM tells the user what went wrong.
- On Telegram download failure, set extracted_text with the error so the
  user sees feedback even when the file never reaches the extraction middleware.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: store extracted document text in workspace memory for search/recall

After document extraction succeeds, write the extracted text to workspace
memory at `documents/{date}/{filename}`. This enables:
- Full-text and semantic search over past uploaded documents
- Cross-conversation recall ("what did that PDF say?")
- Automatic chunking and embedding via the workspace pipeline

Documents are stored with metadata header (uploader, channel, date, MIME type).
Error messages (extraction failures) are not stored — only successful extractions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: CI failures — formatting, unused assignment warning

- Run cargo fmt on document_extraction and agent_loop modules
- Suppress unused_assignments warning on trace_llm_ref (used only
  behind #[cfg(feature = "libsql")])

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review comments — security, correctness, and code quality

Security fixes:
- Remove SSRF-prone download() from DocumentExtractionMiddleware (#13)
- Sanitize filenames in workspace path to prevent directory traversal (#11)
- Pre-check file size before reading in WASM wrapper to prevent OOM (#2)
- Percent-encode file_id in Telegram source URLs (#7)

Correctness fixes:
- Clear image_content_parts on turn end to prevent memory leak (#1)
- Find first *successful* transcription instead of first overall (#3)
- Enforce data.len() size limit in document extraction (#10)
- Use UTF-8 safe truncation with char_indices() (#12)

Robustness & code quality:
- Add 120s timeout to OpenAI Whisper HTTP client (#5)
- Trim trailing slash from Whisper base_url (#6)
- Allow ~/.ironclaw/ paths in WASM wrapper (#8)
- Return error from on_broadcast in Slack/Discord/WhatsApp (#9)
- Fix doc comment in HTTP tool (#4)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: formatting — cargo fmt

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address latest PR review — doc comments, error messages, version bumps

- Fix DocumentExtractionMiddleware doc comment (no longer downloads from source_url)
- Fix error message: "no inline data" instead of "no download URL"
- Log error + fallback instead of silent unwrap_or_default on Whisper HTTP client
- Bump all capabilities.json versions from 0.1.0 to 0.2.0 to match Cargo.toml

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove unsupported profile: minimal from CI workflows [skip-regression-check]

dtolnay/rust-toolchain@stable does not accept the 'profile' input
(it was a parameter for the deprecated actions-rs/toolchain action).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: merge with latest main — resolve compilation errors and PR review nits

- Add version: None to RegistryEntry/InstalledExtension test constructors
- Fix MessageContent type mismatches in nearai_chat tests (String → MessageContent::Text)
- Fix .contains() calls on MessageContent — use .as_text().unwrap()
- Remove redundant trace_llm_ref = None assignment in test_rig
- Check data size before clone in document extraction to avoid unnecessary allocation

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
@ilblackdragon

Copy link
Copy Markdown
Member

All features from #409 are on main:

  • WIT attachment record and on-broadcast export
  • read_attachments() and call_on_broadcast() in wrapper.rs
  • Telegram send_photo/send_document multipart upload
  • HTTP tool save_to parameter with redirect following
  • Message tool /tmp attachment paths
  • Credential env var fallback
  • Discord/Slack/WhatsApp on_broadcast stubs

bkutasi pushed a commit to bkutasi/ironclaw that referenced this pull request Mar 28, 2026
)

* feat: add inbound attachment support to WASM channel system

Add attachment record to WIT interface and implement inbound media
parsing across all four channel implementations (Telegram, Slack,
WhatsApp, Discord). Attachments flow from WASM channels through
EmittedMessage to IncomingMessage with validation (size limits,
MIME allowlist, count caps) at the host boundary.

- Add `attachment` record to `emitted-message` in wit/channel.wit
- Add `IncomingAttachment` struct to channel.rs and re-export
- Add host-side validation (20MB total, 10 max, MIME allowlist)
- Telegram: parse photo, document, audio, video, voice, sticker
- Slack: parse file attachments with url_private
- WhatsApp: parse image, audio, video, document with captions
- Discord: backward-compatible empty attachments
- Update FEATURE_PARITY.md section 7
- Add fixture-based tests per channel and host integration tests

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: integrate outbound attachment support and reconcile WIT types (nearai#409)

Reconcile PR nearai#409's outbound attachment work with our inbound attachment
support into a unified design:

WIT type split:
- `inbound-attachment` in channel-host: metadata-only (id, mime_type,
  filename, size_bytes, source_url, storage_key, extracted_text)
- `attachment` in channel: raw bytes (filename, mime_type, data) on
  agent-response for outbound sending

Outbound features (from PR nearai#409):
- `on-broadcast` WIT export for proactive messages without prior inbound
- Telegram: multipart sendPhoto/sendDocument with auto photo→document
  fallback for files >10MB
- wrapper.rs: `call_on_broadcast`, `read_attachments` from disk,
  attachment params threaded through `call_on_respond`
- HTTP tool: `save_to` param for binary downloads to /tmp/ (50MB limit,
  path traversal protection, SSRF-safe redirect following)
- Message tool: allow /tmp/ paths for attachments alongside base_dir
- Credential env var fallback in inject_channel_credentials

Channel updates:
- All 4 channels implement on_broadcast (Telegram full, others stub)
- Telegram: polling_enabled config, adjusted poll timeout
- Inbound attachment types renamed to InboundAttachment in all channels

Tests: 1965 passing (9 new), 0 clippy warnings

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add audio transcription pipeline and extensible WIT attachment design

Add host-side transcription middleware (OpenAI Whisper) that detects audio
attachments with inline data on incoming messages and transcribes them
automatically. Refactor WIT inbound-attachment to use extras-json and a
store-attachment-data host function instead of typed fields, so future
attachment properties (dimensions, codec, etc.) don't require WIT changes
that invalidate all channel plugins.

- Add src/transcription/ module: TranscriptionProvider trait,
  TranscriptionMiddleware, AudioFormat enum, OpenAI Whisper provider
- Add src/config/transcription.rs: TRANSCRIPTION_ENABLED/MODEL/BASE_URL
- Wire middleware into agent message loop via AgentDeps
- WIT: replace data + duration-secs with extras-json + store-attachment-data
- Host: parse extras-json for well-known keys, merge stored binary data
- Telegram: download voice files via store-attachment-data, add duration
  to extras-json, add /file/bot to HTTP allowlist, voice-only placeholder
- Add reqwest multipart feature for Whisper API uploads
- 5 regression tests for transcription middleware

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: wire attachment processing into LLM pipeline with multimodal image support

Attachments on incoming messages are now augmented into user text via XML tags
before entering the turn system, and images with data are passed as multimodal
content parts (base64 data URIs) to LLM providers. This enables audio transcripts,
document text, and image content to reach the LLM without changes to ChatMessage
serialization or provider interfaces.

- Add src/agent/attachments.rs with augment_with_attachments() and 9 unit tests
- Add ContentPart/ImageUrl types to llm::provider with OpenAI-compatible serde
- Carry image_content_parts transiently on Turn (skipped in serialization)
- Update nearai_chat and rig_adapter to serialize multimodal content
- Add 3 e2e tests verifying attachments flow through the full agent loop

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: CI failures — formatting, version bumps, and Telegram voice test

- Fix cargo fmt formatting in attachments.rs, nearai_chat.rs, rig_adapter.rs,
  e2e_attachments.rs
- Bump channel registry versions 0.1.0 → 0.2.0 (discord, slack, telegram,
  whatsapp) to satisfy version-bump CI check
- Fix Telegram test_extract_attachments_voice: add missing required `duration`
  field to voice fixture JSON

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: bump WIT channel version to 0.3.0, fix Telegram voice test, add pre-commit hook

- Bump wit/channel.wit package version 0.2.0 → 0.3.0 (interface changed with
  store-attachment-data)
- Update WIT_CHANNEL_VERSION constant and registry wit_version fields to match
- Fix Telegram test_extract_attachments_voice: gate voice download behind
  #[cfg(target_arch = "wasm32")] so host functions aren't called in native tests,
  update assertions for generated filename and extras_json duration
- Add @0.3.0 linker stubs in wit_compat.rs
- Add .githooks/pre-commit hook that runs scripts/check-version-bumps.sh when
  WIT or extension sources are staged
- Symlink commit-msg regression hook into .githooks/

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: extract voice download from extract_attachments into handle_message

Move download_voice_file + store_attachment_data calls out of
extract_attachments into a separate download_and_store_voice function
called from handle_message. This keeps extract_attachments as a pure
data-mapping function with no host calls, making it fully testable
in native unit tests without #[cfg(target_arch)] gates.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review comments — security, correctness, and code quality

Security fixes:
- Add path validation to read_attachments (restrict to /tmp/) preventing
  arbitrary file reads from compromised tools
- Escape XML special characters in attachment filenames, MIME types, and
  extracted text to prevent prompt injection via tag spoofing
- Percent-encode file_id in Telegram getFile URL to prevent query injection
- Clone SecretString directly instead of expose_secret().to_string()

Correctness fixes:
- Fix store_attachment_data overwrite accounting: subtract old entry size
  before adding new to prevent inflated totals and false rejections
- Use max(reported, stored_size) for attachment size accounting to prevent
  WASM channels from under-reporting size_bytes to bypass limits
- Add application/octet-stream to MIME allowlist (channels default unknown
  types to this)

Code quality:
- Extract send_response helper in Telegram, deduplicating on_respond and
  on_broadcast
- Rename misleading Discord test to test_parse_slash_command_interaction
- Fix .githooks/commit-msg to use relative symlink (portable across machines)

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add tool_upgrade command + fix TOCTOU in save_to path validation

Add `tool_upgrade` — a new extension management tool that automatically
detects and reinstalls WASM extensions with outdated WIT versions.
Preserves authentication secrets during upgrade. Supports upgrading a
single extension by name or all installed WASM tools/channels at once.

Fix TOCTOU in `validate_save_to_path`: validate the path *before*
creating parent directories, so traversal paths like `/tmp/../../etc/`
cannot cause filesystem mutations outside /tmp before being rejected.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: unify WIT package version to 0.3.0 across tool.wit and all capabilities

tool.wit and channel.wit share the `near:agent` package namespace, so they
must declare the same version. Bumps tool.wit from 0.2.0 to 0.3.0 and
updates all capabilities files and registry entries to match.

Fixes `cargo component build` failure: "package identifier near:agent@0.2.0
does not match previous package name of near:agent@0.3.0"

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: move WIT file comments after package declaration

WIT treats `//` comments before `package` as doc comments. When both
tool.wit and channel.wit had header comments, the parser rejected them
as "doc comments on multiple 'package' items". Move comments after the
package declaration in both files.

Also bumps tool registry versions to 0.2.0 to match the WIT 0.3.0 bump.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: display extension versions in gateway Extensions tab

Add version field to InstalledExtension and RegistryEntry types, pipe
through the web API (ExtensionInfo, RegistryEntryInfo), and render as
a badge in the gateway UI for both installed and available extensions.

For installed WASM extensions, version is read from the capabilities
file with a fallback to the registry entry when the local file has no
version (old installations). Bump all extension Cargo.toml and registry
JSON versions from 0.1.0 to 0.2.0 to keep them in sync.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add document text extraction middleware for PDF, Office, and text files

Extract text from document attachments (PDF, DOCX, PPTX, XLSX, RTF, plain text,
code files) so the LLM can reason about uploaded documents. Uses pdf-extract for
PDFs, zip+XML parsing for Office XML formats, and UTF-8 decode for text files.
Wired into the agent loop after transcription middleware.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: download document files in Telegram channel for text extraction

The DocumentExtractionMiddleware needs file bytes in the attachment `data`
field, but only voice files were being downloaded. Document attachments
(PDFs, DOCX, etc.) had empty `data` and a source_url with a credential
placeholder that only works inside the WASM host's http_request.

Add `download_and_store_documents()` that downloads non-voice, non-image,
non-audio attachments via the existing two-step getFile→download flow and
stores bytes via `store_attachment_data` for host-side extraction.

Also rename `download_voice_file` → `download_telegram_file` since it's
generic for any file_id.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: allow Office MIME types and increase file download limit for Telegram

Two issues preventing document extraction from Telegram:

1. PPTX/DOCX/XLSX MIME types (application/vnd.*) were dropped by the
   WASM host attachment allowlist — add application/vnd., application/msword,
   and application/rtf prefixes.

2. Telegram file downloads over 10 MB failed with "Response body too large" —
   set max_response_bytes to 20 MB in Telegram capabilities.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: report document extraction errors back to user instead of silently skipping

- Bump max_response_bytes to 50 MB for Telegram file downloads
- When document extraction fails (too large, download error, parse error),
  set extracted_text to a user-friendly error message instead of leaving it
  None. This ensures the LLM tells the user what went wrong.
- On Telegram download failure, set extracted_text with the error so the
  user sees feedback even when the file never reaches the extraction middleware.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: store extracted document text in workspace memory for search/recall

After document extraction succeeds, write the extracted text to workspace
memory at `documents/{date}/{filename}`. This enables:
- Full-text and semantic search over past uploaded documents
- Cross-conversation recall ("what did that PDF say?")
- Automatic chunking and embedding via the workspace pipeline

Documents are stored with metadata header (uploader, channel, date, MIME type).
Error messages (extraction failures) are not stored — only successful extractions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: CI failures — formatting, unused assignment warning

- Run cargo fmt on document_extraction and agent_loop modules
- Suppress unused_assignments warning on trace_llm_ref (used only
  behind #[cfg(feature = "libsql")])

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review comments — security, correctness, and code quality

Security fixes:
- Remove SSRF-prone download() from DocumentExtractionMiddleware (nearai#13)
- Sanitize filenames in workspace path to prevent directory traversal (nearai#11)
- Pre-check file size before reading in WASM wrapper to prevent OOM (nearai#2)
- Percent-encode file_id in Telegram source URLs (nearai#7)

Correctness fixes:
- Clear image_content_parts on turn end to prevent memory leak (nearai#1)
- Find first *successful* transcription instead of first overall (nearai#3)
- Enforce data.len() size limit in document extraction (nearai#10)
- Use UTF-8 safe truncation with char_indices() (nearai#12)

Robustness & code quality:
- Add 120s timeout to OpenAI Whisper HTTP client (nearai#5)
- Trim trailing slash from Whisper base_url (nearai#6)
- Allow ~/.ironclaw/ paths in WASM wrapper (nearai#8)
- Return error from on_broadcast in Slack/Discord/WhatsApp (nearai#9)
- Fix doc comment in HTTP tool (nearai#4)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: formatting — cargo fmt

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address latest PR review — doc comments, error messages, version bumps

- Fix DocumentExtractionMiddleware doc comment (no longer downloads from source_url)
- Fix error message: "no inline data" instead of "no download URL"
- Log error + fallback instead of silent unwrap_or_default on Whisper HTTP client
- Bump all capabilities.json versions from 0.1.0 to 0.2.0 to match Cargo.toml

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove unsupported profile: minimal from CI workflows [skip-regression-check]

dtolnay/rust-toolchain@stable does not accept the 'profile' input
(it was a parameter for the deprecated actions-rs/toolchain action).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: merge with latest main — resolve compilation errors and PR review nits

- Add version: None to RegistryEntry/InstalledExtension test constructors
- Fix MessageContent type mismatches in nearai_chat tests (String → MessageContent::Text)
- Fix .contains() calls on MessageContent — use .as_text().unwrap()
- Remove redundant trace_llm_ref = None assignment in test_rig
- Check data size before clone in document extraction to avoid unnecessary allocation

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
drchirag1991 pushed a commit to drchirag1991/ironclaw that referenced this pull request Apr 8, 2026
)

* feat: add inbound attachment support to WASM channel system

Add attachment record to WIT interface and implement inbound media
parsing across all four channel implementations (Telegram, Slack,
WhatsApp, Discord). Attachments flow from WASM channels through
EmittedMessage to IncomingMessage with validation (size limits,
MIME allowlist, count caps) at the host boundary.

- Add `attachment` record to `emitted-message` in wit/channel.wit
- Add `IncomingAttachment` struct to channel.rs and re-export
- Add host-side validation (20MB total, 10 max, MIME allowlist)
- Telegram: parse photo, document, audio, video, voice, sticker
- Slack: parse file attachments with url_private
- WhatsApp: parse image, audio, video, document with captions
- Discord: backward-compatible empty attachments
- Update FEATURE_PARITY.md section 7
- Add fixture-based tests per channel and host integration tests

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: integrate outbound attachment support and reconcile WIT types (nearai#409)

Reconcile PR nearai#409's outbound attachment work with our inbound attachment
support into a unified design:

WIT type split:
- `inbound-attachment` in channel-host: metadata-only (id, mime_type,
  filename, size_bytes, source_url, storage_key, extracted_text)
- `attachment` in channel: raw bytes (filename, mime_type, data) on
  agent-response for outbound sending

Outbound features (from PR nearai#409):
- `on-broadcast` WIT export for proactive messages without prior inbound
- Telegram: multipart sendPhoto/sendDocument with auto photo→document
  fallback for files >10MB
- wrapper.rs: `call_on_broadcast`, `read_attachments` from disk,
  attachment params threaded through `call_on_respond`
- HTTP tool: `save_to` param for binary downloads to /tmp/ (50MB limit,
  path traversal protection, SSRF-safe redirect following)
- Message tool: allow /tmp/ paths for attachments alongside base_dir
- Credential env var fallback in inject_channel_credentials

Channel updates:
- All 4 channels implement on_broadcast (Telegram full, others stub)
- Telegram: polling_enabled config, adjusted poll timeout
- Inbound attachment types renamed to InboundAttachment in all channels

Tests: 1965 passing (9 new), 0 clippy warnings

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add audio transcription pipeline and extensible WIT attachment design

Add host-side transcription middleware (OpenAI Whisper) that detects audio
attachments with inline data on incoming messages and transcribes them
automatically. Refactor WIT inbound-attachment to use extras-json and a
store-attachment-data host function instead of typed fields, so future
attachment properties (dimensions, codec, etc.) don't require WIT changes
that invalidate all channel plugins.

- Add src/transcription/ module: TranscriptionProvider trait,
  TranscriptionMiddleware, AudioFormat enum, OpenAI Whisper provider
- Add src/config/transcription.rs: TRANSCRIPTION_ENABLED/MODEL/BASE_URL
- Wire middleware into agent message loop via AgentDeps
- WIT: replace data + duration-secs with extras-json + store-attachment-data
- Host: parse extras-json for well-known keys, merge stored binary data
- Telegram: download voice files via store-attachment-data, add duration
  to extras-json, add /file/bot to HTTP allowlist, voice-only placeholder
- Add reqwest multipart feature for Whisper API uploads
- 5 regression tests for transcription middleware

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: wire attachment processing into LLM pipeline with multimodal image support

Attachments on incoming messages are now augmented into user text via XML tags
before entering the turn system, and images with data are passed as multimodal
content parts (base64 data URIs) to LLM providers. This enables audio transcripts,
document text, and image content to reach the LLM without changes to ChatMessage
serialization or provider interfaces.

- Add src/agent/attachments.rs with augment_with_attachments() and 9 unit tests
- Add ContentPart/ImageUrl types to llm::provider with OpenAI-compatible serde
- Carry image_content_parts transiently on Turn (skipped in serialization)
- Update nearai_chat and rig_adapter to serialize multimodal content
- Add 3 e2e tests verifying attachments flow through the full agent loop

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: CI failures — formatting, version bumps, and Telegram voice test

- Fix cargo fmt formatting in attachments.rs, nearai_chat.rs, rig_adapter.rs,
  e2e_attachments.rs
- Bump channel registry versions 0.1.0 → 0.2.0 (discord, slack, telegram,
  whatsapp) to satisfy version-bump CI check
- Fix Telegram test_extract_attachments_voice: add missing required `duration`
  field to voice fixture JSON

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: bump WIT channel version to 0.3.0, fix Telegram voice test, add pre-commit hook

- Bump wit/channel.wit package version 0.2.0 → 0.3.0 (interface changed with
  store-attachment-data)
- Update WIT_CHANNEL_VERSION constant and registry wit_version fields to match
- Fix Telegram test_extract_attachments_voice: gate voice download behind
  #[cfg(target_arch = "wasm32")] so host functions aren't called in native tests,
  update assertions for generated filename and extras_json duration
- Add @0.3.0 linker stubs in wit_compat.rs
- Add .githooks/pre-commit hook that runs scripts/check-version-bumps.sh when
  WIT or extension sources are staged
- Symlink commit-msg regression hook into .githooks/

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: extract voice download from extract_attachments into handle_message

Move download_voice_file + store_attachment_data calls out of
extract_attachments into a separate download_and_store_voice function
called from handle_message. This keeps extract_attachments as a pure
data-mapping function with no host calls, making it fully testable
in native unit tests without #[cfg(target_arch)] gates.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review comments — security, correctness, and code quality

Security fixes:
- Add path validation to read_attachments (restrict to /tmp/) preventing
  arbitrary file reads from compromised tools
- Escape XML special characters in attachment filenames, MIME types, and
  extracted text to prevent prompt injection via tag spoofing
- Percent-encode file_id in Telegram getFile URL to prevent query injection
- Clone SecretString directly instead of expose_secret().to_string()

Correctness fixes:
- Fix store_attachment_data overwrite accounting: subtract old entry size
  before adding new to prevent inflated totals and false rejections
- Use max(reported, stored_size) for attachment size accounting to prevent
  WASM channels from under-reporting size_bytes to bypass limits
- Add application/octet-stream to MIME allowlist (channels default unknown
  types to this)

Code quality:
- Extract send_response helper in Telegram, deduplicating on_respond and
  on_broadcast
- Rename misleading Discord test to test_parse_slash_command_interaction
- Fix .githooks/commit-msg to use relative symlink (portable across machines)

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add tool_upgrade command + fix TOCTOU in save_to path validation

Add `tool_upgrade` — a new extension management tool that automatically
detects and reinstalls WASM extensions with outdated WIT versions.
Preserves authentication secrets during upgrade. Supports upgrading a
single extension by name or all installed WASM tools/channels at once.

Fix TOCTOU in `validate_save_to_path`: validate the path *before*
creating parent directories, so traversal paths like `/tmp/../../etc/`
cannot cause filesystem mutations outside /tmp before being rejected.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: unify WIT package version to 0.3.0 across tool.wit and all capabilities

tool.wit and channel.wit share the `near:agent` package namespace, so they
must declare the same version. Bumps tool.wit from 0.2.0 to 0.3.0 and
updates all capabilities files and registry entries to match.

Fixes `cargo component build` failure: "package identifier near:agent@0.2.0
does not match previous package name of near:agent@0.3.0"

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: move WIT file comments after package declaration

WIT treats `//` comments before `package` as doc comments. When both
tool.wit and channel.wit had header comments, the parser rejected them
as "doc comments on multiple 'package' items". Move comments after the
package declaration in both files.

Also bumps tool registry versions to 0.2.0 to match the WIT 0.3.0 bump.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: display extension versions in gateway Extensions tab

Add version field to InstalledExtension and RegistryEntry types, pipe
through the web API (ExtensionInfo, RegistryEntryInfo), and render as
a badge in the gateway UI for both installed and available extensions.

For installed WASM extensions, version is read from the capabilities
file with a fallback to the registry entry when the local file has no
version (old installations). Bump all extension Cargo.toml and registry
JSON versions from 0.1.0 to 0.2.0 to keep them in sync.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add document text extraction middleware for PDF, Office, and text files

Extract text from document attachments (PDF, DOCX, PPTX, XLSX, RTF, plain text,
code files) so the LLM can reason about uploaded documents. Uses pdf-extract for
PDFs, zip+XML parsing for Office XML formats, and UTF-8 decode for text files.
Wired into the agent loop after transcription middleware.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: download document files in Telegram channel for text extraction

The DocumentExtractionMiddleware needs file bytes in the attachment `data`
field, but only voice files were being downloaded. Document attachments
(PDFs, DOCX, etc.) had empty `data` and a source_url with a credential
placeholder that only works inside the WASM host's http_request.

Add `download_and_store_documents()` that downloads non-voice, non-image,
non-audio attachments via the existing two-step getFile→download flow and
stores bytes via `store_attachment_data` for host-side extraction.

Also rename `download_voice_file` → `download_telegram_file` since it's
generic for any file_id.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: allow Office MIME types and increase file download limit for Telegram

Two issues preventing document extraction from Telegram:

1. PPTX/DOCX/XLSX MIME types (application/vnd.*) were dropped by the
   WASM host attachment allowlist — add application/vnd., application/msword,
   and application/rtf prefixes.

2. Telegram file downloads over 10 MB failed with "Response body too large" —
   set max_response_bytes to 20 MB in Telegram capabilities.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: report document extraction errors back to user instead of silently skipping

- Bump max_response_bytes to 50 MB for Telegram file downloads
- When document extraction fails (too large, download error, parse error),
  set extracted_text to a user-friendly error message instead of leaving it
  None. This ensures the LLM tells the user what went wrong.
- On Telegram download failure, set extracted_text with the error so the
  user sees feedback even when the file never reaches the extraction middleware.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: store extracted document text in workspace memory for search/recall

After document extraction succeeds, write the extracted text to workspace
memory at `documents/{date}/{filename}`. This enables:
- Full-text and semantic search over past uploaded documents
- Cross-conversation recall ("what did that PDF say?")
- Automatic chunking and embedding via the workspace pipeline

Documents are stored with metadata header (uploader, channel, date, MIME type).
Error messages (extraction failures) are not stored — only successful extractions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: CI failures — formatting, unused assignment warning

- Run cargo fmt on document_extraction and agent_loop modules
- Suppress unused_assignments warning on trace_llm_ref (used only
  behind #[cfg(feature = "libsql")])

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review comments — security, correctness, and code quality

Security fixes:
- Remove SSRF-prone download() from DocumentExtractionMiddleware (nearai#13)
- Sanitize filenames in workspace path to prevent directory traversal (nearai#11)
- Pre-check file size before reading in WASM wrapper to prevent OOM (nearai#2)
- Percent-encode file_id in Telegram source URLs (nearai#7)

Correctness fixes:
- Clear image_content_parts on turn end to prevent memory leak (nearai#1)
- Find first *successful* transcription instead of first overall (nearai#3)
- Enforce data.len() size limit in document extraction (nearai#10)
- Use UTF-8 safe truncation with char_indices() (nearai#12)

Robustness & code quality:
- Add 120s timeout to OpenAI Whisper HTTP client (nearai#5)
- Trim trailing slash from Whisper base_url (nearai#6)
- Allow ~/.ironclaw/ paths in WASM wrapper (nearai#8)
- Return error from on_broadcast in Slack/Discord/WhatsApp (nearai#9)
- Fix doc comment in HTTP tool (nearai#4)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: formatting — cargo fmt

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address latest PR review — doc comments, error messages, version bumps

- Fix DocumentExtractionMiddleware doc comment (no longer downloads from source_url)
- Fix error message: "no inline data" instead of "no download URL"
- Log error + fallback instead of silent unwrap_or_default on Whisper HTTP client
- Bump all capabilities.json versions from 0.1.0 to 0.2.0 to match Cargo.toml

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove unsupported profile: minimal from CI workflows [skip-regression-check]

dtolnay/rust-toolchain@stable does not accept the 'profile' input
(it was a parameter for the deprecated actions-rs/toolchain action).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: merge with latest main — resolve compilation errors and PR review nits

- Add version: None to RegistryEntry/InstalledExtension test constructors
- Fix MessageContent type mismatches in nearai_chat tests (String → MessageContent::Text)
- Fix .contains() calls on MessageContent — use .as_text().unwrap()
- Remove redundant trace_llm_ref = None assignment in test_rig
- Check data size before clone in document extraction to avoid unnecessary allocation

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor: new First-time contributor risk: medium Business logic, config, or moderate-risk modules scope: channel/wasm WASM channel runtime scope: dependencies Dependency updates scope: tool/builtin Built-in tools size: XL 500+ changed lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants