Repository navigation
Send a question's chosen preview back to Claude Code (annotations) - #24356
Conversation
|
cmux status (one comment, edited in place; each line is updated by the job that owns it) Review App of this head (app.zip, kept 1 day)
Gallery before/after for
States
Pick other states with a Probably yours: CI failed on
Failures:
Matched log linesNot re-run automatically: Written by red: cmux-next checks (god files, concurrency, crash safety, l10n); no finished base run on 714af0c to compare; do not merge until green; see lane-rules |
This comment has been minimized.
This comment has been minimized.
2a3dc10 to
35daeab
Compare
|
Found 45 test failures on Blacksmith runners: Failures
...and 35 more test failures. View all on Blacksmith
|
|
Bead |
|
Orphaned: no activity for 13 h, and no lane holds a lease on it. Adopted by the orphan sweep: merged |
|
|
|
|
|
|
|
Owned by hb-acpmux on big-red · session |
96079b6 to
fb4cc7c
Compare
Claude Code's AskUserQuestion answer may carry annotations (question
text -> {preview, notes}); the agent then reads which mockup the person
picked. The card never sends them, the pane relay refuses them and
acpmux drops them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15d41-f5e8-7631-b4aa-a544e3ef96a8&host=cosmic-gray
The card adds annotations beside a Claude Code answer: for each
single-select item, the chosen option's preview. The pane relay
(Swift and cmux-agent-pane-policy) lets annotations through only beside
answers, keyed by the question's own items, {preview?, notes?} strings.
acpmux takes them only from the person's own connection and with
answers; for Claude Code's AskUserQuestion a preview must be one of
that question's option previews as normalized, the agent gets its own
text back (Claude Code refuses a preview it did not write), and notes
are 1 to 8192 bytes. Any other question drops them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15d41-f5e8-7631-b4aa-a544e3ef96a8&host=cosmic-gray
…red) Codex review: a selected preview over 65,536 UTF-8 bytes rides as an annotation, and both pane relays refuse the whole answer for it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The card sends a chosen preview as an annotation only up to the pane relay's 65,536 UTF-8 bytes (policy.json maximum_annotation_bytes); a longer one is left out, so the answer itself still goes through. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
079ae3e to
6e85bbe
Compare
|
|
Codex review: annotations were checked before the pending lookup, so a second answer from a non-person connection got person_required instead of -32002 with the first answer's answeredBy and outcome. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A losing answer that carries annotations is told who answered first (-32002 with answeredBy and outcome), as any other losing answer is; the person and answers-only rules for annotations still apply to a pending ask. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
|
|
|
…et cover exactly what the person picked (fails: annotations are refused without the person, the daemon verb refuses them, the ticket names no answers) Codex review of #25424: the remote wire dropped a question's annotations (the chosen preview, #24356), and the gesture ticket named no answers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… under the person's proof, and its ticket binds the answers Codex review of #25424: - The remote wire, the session daemon's client and agent-session-permission now carry a question's annotations (Claude Code's chosen preview and notes, #24356) beside its answers. acpmux counts annotations from a connection that is not the person only under the person's decision proof over them: the answers proof gains a seventh part, the annotations' digest, when the answer has annotations. A phone's signed answer still never carries them. - A permission answer's gesture ticket now names the question's answers and annotations, compared by their canonical JSON, so a ticket the page took for one answer never redeems or gets signed for another. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Shipped in:
|
![Fix with [code]smith](https://pr-comments-assets.blacksmith.sh/codesmith/fix-with-codesmith-light-3x.png)
Leo's ask, his words: "ask user question tool and its various states need looking at allegedly there's some sort of vibe-coded thing that agents can do to have prettier ask question situations...? idk but we could look into that/research that then support it..."
Leo-ask: ask-question-states
What was wrong
Claude Code's AskUserQuestion lets an agent attach a preview (a mockup or code sample) to each option. The answer the host sends back may carry
annotations(question text ->{preview?, notes?}), and the agent then reads"Q"="A" selected preview: <mockup>, so it knows which mockup the person picked. cmux sent onlyanswers. The card never sent the chosen preview, the pane relay refused anyannotationsparam, and acpmux would have dropped it anyway.The fix
question/model.tsreply): for a Claude Code ask, each single-select item whose chosen option has a preview addsannotations[prompt] = {preview}. Other answers, multi-selects, the Chief and Codex send nothing new.App.tsxanddirect.tspass it through to_acpmux/permission_respond.AcpmuxPaneMethodsandcmux-agent-pane-policy, from the samepolicy.json:annotationsis a known param. It goes only besideanswers, to a pending question the daemon sent, keyed by that question's own items, and each value is{preview?, notes?}strings of at most 64 KiB.questions::check_annotations,respond_permission_annotated):acpmux-schema.jsondocument the param.Not in this PR: notes UI (the protocol carries notes now; the card doesn't offer a field yet). HTML previews, markdown descriptions, answered and skipped summaries, and MCP elicitation belong to the transcript-parity lane.
Relay analysis
This adds one param to an existing method of the agent pane relay (`_acpmux/permission_respond`). It doesn't add a method, and it changes nothing remote: `RemoteAcpmuxWire` still passes only the option.
Test
Red at the test-only commit 3fc192c, green at the fix:
bun test src/agent-session/acpmux/question/model.test.ts src/agent-session/acpmux/question/record.test.ts(fromwebviews/, on cmux16): red 11 pass / 1 fail ("a Claude answer sends the chosen option's preview"), green 12 pass / 0 fail.questions_annotations*(tests/hub_integration/questions.rs: carried with the agent's own preview text; refused when the preview isn't the agent's, the key or field is unknown, notes are empty or too long, or answers are missing; dropped for a question that isn't Claude's) and the policy cases intests/cases/check.jsonrun on the fleet withcmux-tui-rust-check.sh test. Results are below.cmux-ci compile-check --appat the head: see below.Observed / Not run
Observed: the webview tests above on cmux16. Also at the head on cmux16:
question/,needsYou/andtest/question-card-submit.test.tsxall green,tsc --noEmitclean, andprepush-gate.pyreports no new violations.Not run: a live Claude Code run that reads the annotation back (the fake agent echoes the tool input instead), and the Swift relay unit tests (the shared policy cases cover the same rule in Rust; the Swift harness doesn't read
questionsstate yet).No capture: no visible UI change; the card looks the same and sends one more field.
Gallery: agent-pane.question-card
Changelog
Changed: When you answer an agent's question that has option previews, Claude Code now learns which preview you picked.
Adopted and rebased (2026-10-25)
The stale-pr-reconciler, word for word: "These PRs in your area have no live owner (Leo, 2026-10-25: "these should never have been this stale for this long"). Adopt each:
pr-owner claim N(in that repo), rebase onto its base, an independent review at the new head, then land it".{text, format}objects on base; the card already readspreview.text, andcheck_annotationscompares against the trimmed raw option text, so the agent still gets its own text back. The only conflict was where the new function sits inhub/questions.rs.MAX_ANNOTATION_BYTES, the same aspolicy.jsonmaximum_annotation_bytes), and the answer still goes through. Test first:a preview over the relay's 65,536-byte bound is left out, and the answer still goes(32,768 x "é" is exactly the bound; one byte more, and 65,537 ASCII bytes, are left out).vp fmtreformat of that test only): SHIP at 079ae3e.respond_permission_annotatednow returns(pending, answered_by, annotations), and the request handler keeps base'sanswerer.author.person_requiredinstead of -32002 with the first answer'sansweredBy/outcome. Fixed by moving them to after the lookup. Test first:a_losing_answer_is_told_who_answered_firstnow also sends an annotated losing answer (red at 2eddd47, green at 2551dc9, hub_integration questions/permission/author 40 pass).open-remoteCLI name (cmux-next: one verb to open a persistent remote terminal (Linux VMs too): "open <host> in a bottom split over ssh" #25092).Tests on the rebased branch (fast-checks on Blacksmith)
questions_annotations_carry...and..._refused_and_the_ask_staysfail; policyfull_check_orderfails on the new cases.--test hub_integration questions8 pass,--lib questions5 pass,cmux-agent-pane-policyparity 16 pass; clippy-D warningsand fmt clean.cmux-agent-pane-policy --test typescriptthe_allowlist_is_what_the_page_sendsfails the same way on base de67de9 (_acpmux/mcp/reconnect), so it isn't from this PR.question/,needsYou/,test/question-card-submit.test.tsx46 pass,tsc --noEmitclean.Script tests(release-compile-derived-data.test.sh"bench OTHER_SWIFT_FLAGS", fails on base 5e6cb13); gallery healthagent-pane.composer#file-not-attached(broken on other PRs too).No timers or observers added. No capture: the card looks the same and sends one more field.
Agent session: big-red
🤖 Generated with Claude Code