Repository navigation
feat(chat): render agent HTML visualizations inline in AgentChat - #3370
Conversation
Add a provider-agnostic "visualizations" feature: a coding agent can publish a
self-contained HTML page (chart, table, diagram, collage, mockup) that renders
inline in the AgentChat timeline, above its final reply. Reverse-engineered from
T3 Code's inline HTML renders.
Server:
- New pkg/visualization: themed bootstrap injection (CSS custom properties read
from the URL fragment before first paint, live theme via postMessage, content
height reporting, host-routed links) over the MCP-Apps postMessage protocol.
- html_render MCP tool on the session backend (already wired as the
helix-session context server, so every harness gets it). It stores the
bootstrap-injected page in the session filestore and returns a marker-tagged
reference robust to harness output formatting.
- GET /sessions/{id}/visualizations/{viz_id} serves the page for a sandboxed
iframe (session-authorized; CSP + nosniff). Pages are deleted with the thread.
Frontend:
- visualizationTheme.ts maps the MUI theme to the page's CSS variables and
speaks the postMessage protocol.
- VisualizationFrame renders the page in a sandboxed iframe (no allow-same-origin),
theme via fragment + live postMessage, auto-fit to reported content height,
host-gated link opening.
- InteractionInference detects a completed html_render tool call and renders it
prominently above the reply instead of in the collapsed work log.
Verified end-to-end against the running stack: real html_render MCP call stores
the page; the serving endpoint returns the themed, bootstrap-injected HTML with
the right headers and 403s unauthenticated; headless-Chromium screenshots
confirm both the default and fragment-driven (dark, custom accent) themes paint
before first paint. Go + frontend unit tests and the production build pass. NOT
verified live: the React frame mounting inside the running chat timeline during
a real agent turn (covered by unit tests + build only).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
![helixml-bot[bot]](https://avatars.githubusercontent.com/u/149581110?s=60&v=4){kind=small}
| if err != nil { | ||
| return err | ||
| } | ||
| return c.Options.Filestore.Delete(ctx, folder) |
There was a problem hiding this comment.
GCSStorage.Delete only removes a folder when the path ends in a trailing / (api/pkg/filestore/gcs.go:235), so this bare folder path tries to delete a single non-existent object and the pages are never cleaned up on GCS deployments; append the trailing slash after GetFilestoreUserPath as artifact_handlers.go does for artifact prefixes.
| http.Error(rw, err.Error(), http.StatusNotFound) | ||
| return | ||
| } | ||
| if err := apiServer.authorizeUserToSession(ctx, user, session, types.ActionGet); err != nil { |
There was a problem hiding this comment.
This serving endpoint is the feature's authz and id-regex boundary but has no automated test; add one covering an unauthorized session (403), a traversal-shaped viz_id (400), and the served CSP/nosniff headers.
There was a problem hiding this comment.
Even with a token present, embed API keys get 403 — embedRules in api/pkg/server/auth_embed_key.go has no rule for GET /api/v1/sessions/{id}/visualizations/{viz_id}; add a scopeSession entry so embedded chat can load visualizations.
Reviewed: a0b7a82
| // app uses the BFF session cookie and needs nothing extra. | ||
| const embedAccessToken = (): string | null => { | ||
| if (typeof window === 'undefined') return null | ||
| return new URLSearchParams(window.location.search).get('access_token') |
There was a problem hiding this comment.
useApi.ts strips access_token from the URL at module init, so this always returns null and embed-context iframes request with no credentials and get 401 — visualizations never render in embedded chat; export the embed token useApi already captured and read it here, or fetch the page via the authenticated client and render it with srcdoc.
Org bot instances could not visualize: html_render lived on helix-session,
which ApplyBotInstanceProfile strips unless the profile lists it (the default
keeps only chrome-devtools). A live chief-of-staff instance had served
context servers [chrome-devtools, helix] and fell back to writing a local
HTML file and screenshotting it.
- Move html_render to a dedicated visualization MCP backend, wired into
every config as helix-viz and always kept by ApplyBotInstanceProfile, so
bots visualize without also gaining session navigation. The session
backend is back to its original form.
- Allow the visualization backend for bot-instance keys, bound to the key's
own session_id. Required atomically: the sandbox readiness gate probes
every remote context server and fails Zed launch on 401/403.
- Serve pages at GET /sessions/{id}/visualization?viz_id= and allow it for
embed keys scoped to their session; the iframe now gets the embed token
captured by useApi (it is stripped from the URL at load).
- Treat helix-viz as Helix-owned in the settings-sync-daemon.
Verified live: a new chief-of-staff instance (minimal profile, Qwen) served
[chrome-devtools, helix-viz], called helix-viz_html_render unprompted, and
the chart rendered inline in the org chat. With the bot's own key:
readiness OPTIONS 404 (pass), tools/list = [html_render], cross-session
403, helix-session still 403.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
In light mode the page painted MUI's #fff while the chat column is #fafafa, so every visualization read as a box. Surface tokens now come from getChatColors; semantic colours stay on the MUI palette. Measured live: light #fafafa/#fafafa, dark #0a0a0a/#0a0a0a (surface/page). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
| if err != nil { | ||
| return err | ||
| } | ||
| return c.Options.Filestore.Delete(ctx, folder) |
There was a problem hiding this comment.
GCSStorage.Delete treats a path without a trailing slash as a single object (api/pkg/filestore/gcs.go:235), so this folder cleanup is a no-op error on GCS deployments and the pages are never deleted with the thread; append a trailing slash to the folder path (the filesystem backend ignores it).
| http.Error(rw, err.Error(), http.StatusNotFound) | ||
| return | ||
| } | ||
| if err := apiServer.authorizeUserToSession(ctx, user, session, types.ActionGet); err != nil { |
There was a problem hiding this comment.
This endpoint is the feature's authz and viz-id boundary but has no automated test; add a handler test covering a foreign session (403), a traversal-shaped viz_id (400), and the served CSP/nosniff headers.
1 participant
Note
Medium Risk
Adds an agent-published inline-HTML feature spanning the MCP tool surface, the session filestore, a new session-scoped serving endpoint and the auth rules for bot-instance and embed keys; the realistic failure modes are auth-boundary mistakes on the new endpoint and storage/cleanup gaps in the session filestore.
Overview
Adds an
html_renderMCP tool on a dedicated visualization backend (/api/v1/mcp/visualization), wired into every agent's Zed config as thehelix-vizcontext server — spec tasks and ordinary chat viaGenerateZedMCPConfig, org bots becauseApplyBotInstanceProfilenow always keepshelix-viz. The tool validates input (2MB cap, cleaned title, clamped height), injects a theme/bootstrap script into the page<head>(CSS custom properties, live theme via postMessage, content-height reporting, host-routed links over the MCP-Apps protocol), stores the page in the session's filestore undervisualizations/<viz_id>.html, and returns a marker-tagged JSON reference (HELIX_VISUALIZATION_V1 {...}) the frontend can find regardless of harness wrapping.Adds
GET /api/v1/sessions/{id}/visualization?viz_id=…on the subRouter, authorizing viaauthorizeUserToSession(ActionGet)(bearer /access_tokencookie / query, mirroringgetSession) and serving the stored page astext/htmlwithnosniff, a sandbox CSP and a short private cache;deleteSessionbest-effort removes the session's visualizations folder. Auth rules ship with the config change: bot-instance keys may reach the visualization backend only for their own session, embed keys may GET/visualizationfor their own session only, and the settings-sync-daemon treatshelix-vizas Helix-owned.On the frontend,
InteractionInferencedetects a completedhtml_rendertool call carrying the marker and renders the page prominently above the reply through a newVisualizationFramesandboxed iframe (noallow-same-origin), handing over the active MUI theme via URL fragment plus postMessage, auto-fitting to the reported content height and gating link opens on frame focus plus user activation. A design doc records the availability matrix and follow-ups.Reviewed by helixml-bot for commit 20e5b69.
What
Adds a provider-agnostic visualizations feature: a coding agent (Zed Agent, Claude Code, Codex, Qwen, Goose, …) can publish a self-contained HTML page — a chart, table, diagram, collage, or mockup — that renders inline in the AgentChat timeline, above its final reply.
Reverse-engineered from T3 Code's inline HTML renders (pingdotgg/t3code PRs #15968 and #16283), adapted to Helix's MCP + filestore + MUI-theme stack.
How it works
html_renderMCP tool on a dedicated visualization MCP backend, wired into every agent config as thehelix-vizcontext server — spec tasks, ordinary chat and org bots (it is always kept byApplyBotInstanceProfile, so a minimal bot gets visualizations without also getting session navigation). It takes a complete HTML document + title (+ optional height).<head>(api/pkg/visualization): CSS custom properties read from the URL fragment before first paint, live theme updates viapostMessage, content-height reporting, and host-routed link clicks — over the MCP-Apps postMessage protocol.HELIX_VISUALIZATION_V1 {json}) that the frontend finds regardless of how a harness wraps tool output.GET /api/v1/sessions/{id}/visualization?viz_id=…serves it for a sandboxed iframe (allow-scripts allow-forms, noallow-same-origin→ opaque origin, can't touch app session/storage). Session-authorized; CSP +nosniff. Pages are deleted with the thread.VisualizationFrame) renders the page inline, hands it the active MUI theme (fragment + live postMessage), auto-fits to the reported content height, and gates link opens on frame focus + user activation.InteractionInferencedetects a completedhtml_rendercall and renders it prominently above the reply, not in the collapsed work log.Scope
v1 ships
html_render(inline display). T3'shtml_preview(headless-Chrome self-check) is deliberately out of scope — the bootstrap's livesize-changedreporting means the client fits the frame without the server pre-measuring, so preview isn't needed for correct rendering. See the design doc for details and follow-ups (agent prompt nudge,html_preview).Auth surfaces touched
/api/v1/mcp/visualizationonly for their ownsession_id. This must ship with the config change: the sandbox readiness gate OPTIONS-probes every remote context server and fails Zed launch on 401/403.GET /api/v1/sessions/{id}/visualizationfor their own session only; the iframe carries the embed tokenuseApicaptured at load.Verification
Live, on a real org bot (
unmanned-orgchief-of-staff, minimalchrome-devtools-only instance profile, Qwen harness): a new instance chat asking for "a bar chart" served context servers[chrome-devtools, helix-viz]; the agent calledhelix-viz_html_renderunprompted; the chart rendered inline above its reply in the org chat (headless Chromium, real page). Page background measured equal to the chat canvas in light (#fafafa) and dark (#0a0a0a).With that bot's own sandbox key: readiness OPTIONS → 404 (the gate's pass status),
tools/list→[html_render]only, cross-session → 403,helix-session→ still 403.Serving: 200
text/htmlwith sandbox CSP +nosniff; traversal id → 400; unauthenticated → 403.Tests + build: Go —
pkg/visualization, thehtml_renderhandler suite against a real filesystem filestore,ApplyBotInstanceProfile(helix-viz survives every profile), bot-instance key allow/deny, embed key allow/deny, settings-sync-daemon. Frontend — theme mapping, marker parsing, protocol readers, timeline detection;yarn tsc+yarn build.NOT verified: iOS/WebKit rendering of the iframe, and the embedded (
/embed/…) page end to end — the embed-key rule and token path are unit-tested and the token path ran via?access_token=on the normal chat page.What's needed to roll out
sandbox-versions.txtbump, no new service.helix-vizis kept for every instance profile, including the defaultchrome-devtools-only one.helix-vizas Helix-owned. That only stops a stale on-disk entry from overriding the API's config; it takes effect with the next./stack build-ubuntuand the feature works without it.sessions/<id>/visualizations/) and removed when the session is deleted.🤖 Generated with Claude Code