Skip to content

feat(studio): Support images in chat - #364

Merged
steramae-nvidia merged 8 commits into
mainfrom
steramae/image-chat
Jun 22, 2026
Merged

steramae-nvidia merged 8 commits into
mainfrom
steramae/image-chat

Conversation

@steramae-nvidia

@steramae-nvidia steramae-nvidia commented Jun 16, 2026 •

Copy link
Copy Markdown
Contributor
Screenshot 2026-06-15 at 3 55 05 PM Screenshot 2026-06-16 at 2 42 00 PM

Summary by CodeRabbit

  • New Features
    • Added image attachment support to assistant chat, with UI enabled only for vision-capable models and controlled by a new opt-in prop.
    • Introduced reusable message and composer components for assistant chat rendering.
  • Documentation
    • Updated AssistantChat docs to clarify supported image multimodal behavior and narrow “not yet supported” attachment types.
  • Tests
    • Expanded coverage for image/vision behavior, message serialization, and edit-mode rendering and interactions.

Signed-off-by: Sean Teramae <steramae@nvidia.com>
Signed-off-by: Sean Teramae <steramae@nvidia.com>
@steramae-nvidia
steramae-nvidia requested review from a team as code owners June 16, 2026 21:51
@github-actions github-actions Bot added the feat label Jun 16, 2026
@coderabbitai

coderabbitai Bot commented Jun 16, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 0e048c8f-aad8-4314-9699-ae28d03b3801

📥 Commits

Reviewing files that changed from the base of the PR and between f53b42e and d29913a.

📒 Files selected for processing (4)
  • web/packages/common/src/components/AssistantChat/ComposerAttachments.tsx
  • web/packages/common/src/components/AssistantChat/index.test.tsx
  • web/packages/common/src/components/AssistantChat/messageUtils.ts
  • web/packages/common/src/components/AssistantChat/useAssistantChatRuntime.ts
💤 Files with no reviewable changes (3)
  • web/packages/common/src/components/AssistantChat/ComposerAttachments.tsx
  • web/packages/common/src/components/AssistantChat/useAssistantChatRuntime.ts
  • web/packages/common/src/components/AssistantChat/messageUtils.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • web/packages/common/src/components/AssistantChat/index.test.tsx

📝 Walkthrough

Walkthrough

Adds image attachment support to AssistantChat with a new enableImageAttachments prop (default true) AND-gated with modelSupportsImageAttachments. Multimodal message utilities convert image parts to OpenAI image_url content. New shared components (AssistantComposer, UserEditComposer, AssistantMessage, UserMessage, ComposerAttachmentsRow) are extracted and threaded throughout the component tree.

Changes

AssistantChat Image Attachments

Layer / File(s) Summary
Types, multimodal utilities, and model detection
types.ts, messageUtils.ts
Adds UserMessageContentPart, modelSupportsImageAttachments, isImagePart, getMessageImageParts, and updated getUserMessageContent/OpenAI serialization for image_url multimodal content. Expands AssistantChatThreadProps, AssistantChatMessageContentProps, MessageRenderProps, and AssistantChatProps with enableImageAttachments.
Multimodal message utility tests
messageUtils.test.ts
DATA_URL fixture, multimodal getOpenAIMessages assertions, getUserMessageContent text+image composition tests, and modelSupportsImageAttachments vision detection tests.
Runtime adapter wiring and message handling
useAssistantChatRuntime.ts
Conditionally wires SimpleImageAttachmentAdapter via enableImageAttachments option; updates handleNewMessage/handleEdit to use getUserMessageContent for multimodal content arrays.
Shared message content, actions, and attachment UI
AssistantChatMessageContent.tsx, messageActions.tsx, ComposerAttachments.tsx
Adds AssistantChatMessageContent (Text/Image/tools renderers + error banner), CopyAction with conditional icon state, and ComposerAttachmentsRow/ComposerAttachmentChip with useImagePreviewUrl hook for transient File previews and remove controls.
AssistantMessage, UserMessage, UserEditComposer
AssistantMessage.tsx, UserMessage.tsx, UserEditComposer.tsx
Extracts shared message components: AssistantMessage with running skeleton and regenerate/copy action bar; UserMessage with edit/copy actions; UserEditComposer with conditional attachment row and enter-to-submit.
AssistantComposer with image attachment slot
AssistantComposer.tsx
New shared composer with optional slotComposerStart, conditional image attachment button/row, paste-to-attachment input, reset button, and Stop vs Submit switching.
AssistantChatThread refactor
AssistantChatThread.tsx
Removes 270+ lines of local UI; imports shared components; threads enableImageAttachments into UserEditComposer and AssistantComposer; re-exports AssistantChatThreadAttributes from types.
AssistantChat entry point gate
index.tsx
Adds enableImageAttachments prop (default true), computes imageAttachmentsEnabled via modelSupportsImageAttachments, and forwards to runtime and thread.
Integration tests and documentation
index.test.tsx, README.md
Adds Add image button visibility/state tests, user message image rendering test, edit-mode image seeding test; upgrades userEvent interactions to userEvent.setup(); updates README supported/unsupported feature lists.

Possibly related PRs

Suggested reviewers

  • dmariali
  • htolentino-nvidia
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly describes the main change: adding image support to the chat interface. It's specific, concise, and aligns with all file modifications centered on enabling vision/image attachment functionality.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch steramae/image-chat

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@web/packages/common/src/components/AssistantChat/index.tsx`:
- Line 9: Replace the relative import path in the import statement for
modelSupportsImageAttachments with an absolute alias path. The current import
uses './messageUtils' which violates the absolute alias path requirement for
web/**/*.{ts,tsx,js,jsx} files. Convert this relative import to use the
appropriate absolute alias path configured in the project (typically using path
aliases like `@common` or similar based on tsconfig configuration).

In `@web/packages/common/src/components/AssistantChat/messageUtils.test.ts`:
- Around line 6-10: Replace the relative import path in the import statement for
getOpenAIMessages, getUserMessageContent, and modelSupportsImageAttachments with
an absolute alias path instead of the relative './messageUtils' path. Follow the
project's established absolute import alias convention for web packages to
maintain consistency with other imports in the codebase.

In `@web/packages/common/src/components/AssistantChat/README.md`:
- Around line 69-70: The documentation for image attachments at lines 69-70 in
the README.md file is incomplete and misleading. Update the wording to clarify
that image attachments are enabled by both the enableImageAttachments prop AND
the modelSupportsImageAttachments function check, not by the prop alone. Rewrite
the bullet point to accurately reflect the full runtime condition that gates the
image attachment feature in the AssistantChat component.

In `@web/packages/common/src/components/AssistantChat/types.ts`:
- Around line 129-132: The JSDoc comment for the enableImageAttachments property
specifies `@default` false, but the actual implementation in index.tsx defaults
this property to true. Update the `@default` annotation in the JSDoc comment for
enableImageAttachments in the types.ts file to reflect the true default value
used in the implementation.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 34be087d-0b35-46f8-b6ef-c9fdba48d520

📥 Commits

Reviewing files that changed from the base of the PR and between 8dc552f and 7c8dc3d.

📒 Files selected for processing (12)
  • services/studio/src/nmp/studio/env_mappings.py
  • web/packages/common/src/components/AssistantChat/AssistantChatMessages.tsx
  • web/packages/common/src/components/AssistantChat/AssistantChatThread.tsx
  • web/packages/common/src/components/AssistantChat/AssistantComposer.tsx
  • web/packages/common/src/components/AssistantChat/ComposerAttachments.tsx
  • web/packages/common/src/components/AssistantChat/README.md
  • web/packages/common/src/components/AssistantChat/index.test.tsx
  • web/packages/common/src/components/AssistantChat/index.tsx
  • web/packages/common/src/components/AssistantChat/messageUtils.test.ts
  • web/packages/common/src/components/AssistantChat/messageUtils.ts
  • web/packages/common/src/components/AssistantChat/types.ts
  • web/packages/common/src/components/AssistantChat/useAssistantChatRuntime.ts

Comment thread web/packages/common/src/components/AssistantChat/index.tsx Outdated
Comment thread web/packages/common/src/components/AssistantChat/messageUtils.test.ts Outdated
Comment thread web/packages/common/src/components/AssistantChat/README.md Outdated
Comment thread web/packages/common/src/components/AssistantChat/types.ts
@github-actions

github-actions Bot commented Jun 16, 2026 •

Copy link
Copy Markdown
Contributor
Suite Lines Covered Line Rate Branch Rate
Unit Tests 20036/26510 75.6% 60.9%
Integration Tests 11692/25282 46.2% 19.8%

steramae-nvidia and others added 2 commits June 16, 2026 16:19
- Use absolute alias path for messageUtils import in index.tsx and test
- Fix @default annotation: enableImageAttachments defaults to true not false
- Update README to reflect that image attachments require both
  enableImageAttachments=true and a vision-capable model

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Sean Teramae <steramae@nvidia.com>
Signed-off-by: Sean Teramae <steramae@nvidia.com>
Comment thread web/packages/common/src/components/AssistantChat/AssistantChatMessages.tsx Outdated
Comment thread web/packages/common/src/components/AssistantChat/ComposerAttachments.tsx Outdated
Signed-off-by: Sean Teramae <steramae@nvidia.com>
Signed-off-by: Sean Teramae <steramae@nvidia.com>
Signed-off-by: Sean Teramae <steramae@nvidia.com>
Signed-off-by: Sean Teramae <steramae@nvidia.com>
@steramae-nvidia
steramae-nvidia added this pull request to the merge queue Jun 22, 2026
Merged via the queue into main with commit df4e0af Jun 22, 2026
51 checks passed
@steramae-nvidia
steramae-nvidia deleted the steramae/image-chat branch June 22, 2026 17:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants