Skip to content

fix(web): render agent-authored HTML views in chat again - #152

Merged
tusharbhardwaj-bk merged 1 commit into
bkmainfrom
t3code/agent-views-html
Sep 1, 2026
Merged

fix(web): render agent-authored HTML views in chat again#152
tusharbhardwaj-bk merged 1 commit into
bkmainfrom
t3code/agent-views-html

Conversation

@tusharbhardwaj-bk

@tusharbhardwaj-bk tusharbhardwaj-bk commented Sep 1, 2026

Copy link
Copy Markdown

Problem

A t3_show_ui call left an ordinary collapsed tool row in the transcript and rendered nothing. Reported against thread bc541f47-d2d9-4a9d-91ba-524fe4c86ca9, where an agent sent 6 HTML views twice — the second time with all content statically baked in, on the theory that scripts were being blocked. Neither batch appeared.

The server side was never at fault. All 12 renders are stored correctly, and the render handle reaches the timeline row:

2026-08-31T19:00:10Z  html   6655  Option 6 — Heatmap Scanner
2026-08-31T18:59:44Z  html  10882  Option 5 — Swipeable Metric Pages
2026-08-31T18:58:55Z  html   7126  Option 4 — Compact List + Bottom-Sheet Drill-down
...  (12 rows, all kind=html)

The client refused to mount them. #145 disabled agent views behind a blanket kill switch:

export const AGENT_UI_SURFACES_RUNTIME_ENABLED = false;

AgentUiSurfaceRow short-circuits on it and returns the plain tool row. bkt3 is running 0436d9d6, which contains it.

Fix

The concern behind #145 was real but URL-only: a framed collaboration app can decline to join the room named by the URL and render unrelated origin-local state instead — a box that looks right and shows the wrong thing. The flag also took out agent-authored HTML, which has none of that problem. The document is what the agent produced, and it mounts from srcDoc in an unmodified sandbox="allow-scripts" iframe, so it has an opaque origin and can reach neither T3 nor the network as the signed-in user. AgentUiRenderFrame already blocked URL renders independently, so the global flag was over-broad and redundant — and it left the Experiments toggle permanently disabled.

This narrows the gate to the kind it was justified for. isAgentUiSurfaceRenderable() allows html, blocks url. A url handle still leaves the ordinary tool row, and the frame keeps its URL-blocked message for a stored render whose kind disagrees with the record.

Because the renders are already in the database, the views in the reported thread appear retroactively once this deploys — nothing needs re-sending.

Evidence

Before: in the reported thread every t3_show_ui call renders as a bare collapsed tool row — the wrench icon, the truncated JSON argument, and a chevron — with no view box beneath it. (Screenshot is in the originating T3 session; this agent has no way to upload an attachment.)

I have not verified this in a browser; the "after" is covered by tests, not a screenshot. New regression guard asserts an HTML surface mounts an iframe carrying sandbox="allow-scripts" and the agent's markup in srcdoc — the coverage whose absence let #145 disable the working half of the feature silently.

  • vp test run apps/web/src/fork/ MessagesTimeline.agentUi.logic.test.ts ActivityPayloadProjection.agentUi.test.ts — 59 passed / 10 files
  • vp run typecheck (@t3tools/web) — clean
  • Cherry-picked onto 0436d9d6, the SHA bkt3 currently runs, and re-verified there

Note on the base

This targets bkmain directly at the maintainer's explicit request, skipping the usual expbkt3 soak. All touched source files are byte-identical between bkmain and expbkmain, so the change is isolated from the 9 mobile commits currently staged in expbkmain.

Claude Opus 5 via T3 Code.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

A t3_show_ui call stored its document server-side and left an ordinary
collapsed tool row in the transcript, rendering nothing. The renders were
stored correctly and the handle reached the timeline; the client refused to
mount them.

PR #145 disabled agent views behind a blanket
AGENT_UI_SURFACES_RUNTIME_ENABLED = false. The concern behind it was real but
URL-only: a framed collaboration app can decline to join the room named by the
URL and render unrelated origin-local state instead. The flag also took out
agent-authored HTML, which has none of that problem — the document is what the
agent produced, and it mounts from srcDoc in an unmodified allow-scripts
sandbox with an opaque origin. AgentUiRenderFrame already blocked URL renders
on their own, so the global flag was both over-broad and redundant, and it left
the Experiments toggle permanently disabled.

Narrow the gate to the kind it was justified for: isAgentUiSurfaceRenderable()
allows html and blocks url. A url handle still leaves the ordinary tool row,
and the frame keeps its URL-blocked message for a stored render whose kind
disagrees with the record. Re-enables the Experiments toggle with a truthful
description.

Adds the regression guard that was missing: an HTML surface mounts an iframe
carrying sandbox="allow-scripts" and the agent's markup in srcdoc.

Claude Opus 5 via T3 Code.
@github-actions github-actions Bot added size:M vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. labels Sep 1, 2026
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ℹ️ No successful main baseline artifact is available yet. This run establishes the initial measurement.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire 13.5 KiB 15.1 KiB
Codex Thread snapshot wire 7.2 KiB 7.3 KiB
Codex Live turn WebSocket wire 6.4 KiB 7.8 KiB
Codex Live turn WebSocket decoded 55.6 KiB 66.4 KiB
Codex Live turn messages 10 21
Claude Total thread wire 13.6 KiB 15.1 KiB
Claude Thread snapshot wire 7.2 KiB 7.3 KiB
Claude Live turn WebSocket wire 6.4 KiB 7.8 KiB
Claude Live turn WebSocket decoded 56.5 KiB 66.4 KiB
Claude Live turn messages 11 21

Baseline: unavailable · PR result: 2b5b30c · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 110.4 KiB
  • Claude decoded thread snapshot: 111.1 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

@tusharbhardwaj-bk
tusharbhardwaj-bk merged commit c1331e7 into bkmain Sep 1, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant