feat(code): encode multi-select ask_user answers as JSON arrays - #5660
Merged
Mason Daugherty (mdrxy) merged 11 commits intoAug 20, 2026
Conversation
Multi-select answers were encoded by joining selected values with ", ", which forced a comma ban on choice values and custom Other text — enforced by a `ValueError` that escaped `ToolNode` and killed the agent turn. The answer is now a self-delimiting JSON array (`["a", "b"]`, `[]` when untouched), so commas/quotes/newlines in values round-trip exactly and the ban is deleted. Validation failures raise `ToolException` with `handle_tool_error=True` set on the tool, so a malformed payload comes back as an error `ToolMessage` the model can act on instead of a fatal turn error.
github-actions
Bot
added
dcode
`_same_turn_user_answers` tested emptiness with `answer.strip()`. That was correct when an untouched optional question encoded as `""`, but a `multi_select` now encodes as the truthy string `[]`, so a question the user declined to answer entered `same_turn_user_answers` paired with something that reads like an answer. `_CLASSIFIER_POLICY` tells the classifier that unselected choices are omitted and grant nothing, which was no longer true. Two notions of "empty" had drifted apart across the wire: the TUI grew `answer_is_empty` for exactly this reason while the authorization side kept the bare `.strip()`. Add `ask_user_answer_is_empty` next to the codec as the single definition and call it from both sides. The skip now runs after the question shape guard rather than before it. Those guards are unreachable today, and taking them first is the fail-closed order. This also stops a declined question from consuming the `MAX_ASK_USER_AUTHORIZATION_QUESTION_TOTAL_CHARS` budget, whose overflow discards every authorization row for the turn.
Deleting `_ask_user.handle_tool_error = True` left the whole suite green. The existing tool tests call `ask_tool.func` directly, which bypasses `BaseTool.run` and so never reaches the error handling, and the type assertion on `_validate_questions` pins only the raise site. Drive a real `ToolNode` in a compiled graph and assert the rejected call comes back as an error `ToolMessage` instead of escaping the node. Both new tests fail with the attribute removed. Also correct the docstring on `test_validation_errors_are_tool_exceptions`: the conversion happens in `BaseTool.run`, not in `ToolNode`, whose default handler converts only `ToolInvocationError`.
The inline question widget is unmounted once answered, so that row is the only place the answers stay visible. It renders the transcript literally, so JSON encoding put the wire format on screen: `A: ["Boston, MA", "Austin"]` where the row used to read `A: Boston, MA, Austin`, and a multi-line custom answer collapsed to a literal `\n` on one line. The comma ban this PR removed was justified as buying legibility of the joined answer, so that cost needed somewhere to go. Add `render_ask_user_transcript_for_display`, which rebuilds the transcript with each decodable multi-select answer expanded to one value per line. Recovering the answers means splitting a transcript that `format_ask_user_transcript` warns is not unambiguously decodable, so it anchors on the known question text — the mitigation that docstring prescribes — and returns `None` on any mismatch, leaving the caller its literal rendering. Placeholders such as `(cancelled)` are not JSON, so they still show verbatim. Display only; nothing here feeds a trust decision.
`_QuestionWidget.answer_is_empty` reimplemented the multi-select rule that now lives in `ask_user_answer_is_empty`; delegate instead, so the widget and the authorization path cannot drift again. `on_ask_user_text_area_submitted` still tested `get_answer().strip()`. That is reachable for a multi-select, where `[]` is truthy — correct today only because `validate_for_submit` runs first and catches the required-and-empty case. Reordering those two would have made a required empty multi-select confirm silently. Use `answer_is_empty` so it holds by construction. Also trim the `[]`-is-truthy rationale in the submit loop, which restated the docstring it points at.
`handle_tool_error = True` means a malformed `questions` payload no longer kills the turn — the model gets an error `ToolMessage` and retries. Nothing on that path logged, so a model looping on the same bad payload was invisible until the recursion limit tripped, leaving an error that names none of the seven real causes. Log once where the tool calls the validator, matching what `_parse_answers` already does for every malformed-payload branch. Widen the codec tests to pin behavior that was already correct but untested: brackets inside a value (the one character class that collides with the encoding's own delimiters, including a value that is itself a JSON array), and decode rejection of the empty string, JSON scalars, nested arrays, and a null element. The empty string matters most — it is the pre-JSON encoding of an untouched question and is still synthesized on `textual_adapter`'s error and cancel paths, so a lenient decode would quietly restore the old semantics. Also retire a stale contrast in `test_allows_comma_in_multiple_choice_value`, which distinguished single-selection answers from multi-select on comma handling. Both accept commas now.
![open-swe[bot]](https://avatars.githubusercontent.com/in/1395372?s=60&v=4){kind=small}
…nscript A text answer can legitimately contain a later `Q: ...` block verbatim. Anchoring on the first occurrence of the next block folded the real multi-select block into the preceding answer, so the expanded row showed the quoted `["fake"]` as that answer. A separator that occurs more than once now fails the parse and the transcript renders literally.
…ct-json-answers # Conflicts: # libs/code/deepagents_code/ask_user.py # libs/code/tests/unit_tests/test_ask_user_middleware.py
Corrections to the JSON multi-select encoding: documentation that contradicted the code, one display placeholder with two meanings, a dead guard, a duplicate log, and unattributable drops of consent evidence. Behavior: - An unselected `multi_select` now displays as `(nothing selected)` via a new display-only `ASK_USER_NOTHING_SELECTED`. It previously reused `ASK_USER_NO_ANSWER`, whose docstring promises that constant marks a *missing* answer and never a deliberately blank one. - `_same_turn_user_answers` logs when it withholds an undecodable `multi_select` answer from the authorization evidence. Only a non-TUI client can produce one; withholding is fail-closed but costs the user an authorization they gave, so the drop is now attributable. The answer text is not logged. - `_CLASSIFIER_POLICY` states that a multi-select answer arrives as a JSON array, so the classifier compares values rather than brackets and quotes. - The TUI row only attempts the display re-render when a `multi_select` is actually present, and debug-logs when it cannot unpack one. - Dropped the duplicate `logger.warning` in the `ask_user` tool body: `_tool_arg_validation_on_error` already logs the rejection with `exc_info`, and its own comment claims to be the single record. Dead code: - Removed `if position != len(transcript)` from `render_ask_user_transcript_for_display`. It was unreachable — the final loop iteration always sets `position` to the end — and it advertised a trailing-content rejection that does not exist: with two or more questions the junk lands in the last answer and is re-emitted. Corrected comments, each verified against the code: - `format_ask_user_transcript` no longer claims the TUI never parses the transcript, and no longer says the JSON encoding "buys nothing here" — it does close the block-forgery hazard, this function just cannot rely on it. - `encode_multi_select_answer` drops "compact" (the default separators are `", "`) and scopes the no-newline guarantee: `ensure_ascii=False` passes U+2028/U+2029 through literally. - `ask_user_answer_is_empty` says the TUI re-prompts a *required* question, and documents that `(cancelled)`/`(error: ...)` read as empty for `multi_select` only. - The `auto_mode` skip comment names the real mechanisms: the char budget rejects the whole row set, and `rows[-20:]` is what evicts. - Documented `decode_multi_select_answer`'s `str` precondition, the `end + 2` uniqueness re-search, the trailing-content and duplicate-question fallbacks, that the per-answer receipt budget is measured on the encoded form, and the `["a", "b"]` vs `["a\nb"]` display ambiguity. - Fixed `\\n` rendering as two characters inside an r-string docstring. - One phrasing for the permitted characters across all three model-facing sites, plus the `multiple_choice` counterpart: that value is returned bare and unescaped, so it must stay single-line. Tests for gaps where the code could be broken with the suite still green: - The emptiness skip running before the char budget, so a declined question cannot discard a real affirmative. - An undecodable `multi_select` answer being withheld and logged. - A benign quoted `Q:`/`A:` block still unpacking its neighbour, the over-rejection side the uniqueness rule trades against. - Trailing content being re-emitted, and three identically worded questions falling back to literal rendering. - The preview branch staying a summary with `multi_select` args. - A required text answer of literally `[]` not reading as empty. - JSON escaping pushing an answer past the receipt cap.
Mason Daugherty (mdrxy)
deleted the
mdrxy/code/multi-select-json-answers
branch
August 20, 2026 15:58
Mason Daugherty (mdrxy)
pushed a commit
that referenced
this pull request
Aug 20, 2026
> [!CAUTION] > Merging this PR will automatically publish to **PyPI** and create a **GitHub release**. For the full release process, see [`.github/RELEASING.md`](https://github.com/langchain-ai/deepagents/blob/main/.github/RELEASING.md). --- _Release notes preview: keep this section in sync with the package `CHANGELOG.md`. Publish reads the merged CHANGELOG via `release.yml`, not this PR description — keep them aligned anyway so the PR stays an accurate historical record for reviewers and anyone returning later._ --- ## [0.1.59](deepagents-code==0.1.58...deepagents-code==0.1.59) (2026-08-20) ### Features - Added support for `managed_config.toml` configuration ([#5604](#5604)) - Multi-select `ask_user` answers are now encoded as JSON arrays ([#5660](#5660)) - Made teardown usage stats configurable ([#5696](#5696)) - Footer pickers now open on click ([#5674](#5674)) - Replaced Gemini 3.6 Flash with Gemini 3.7 Flash ([#5681](#5681)) ### Bug fixes - Made tool argument validation errors recoverable ([#5659](#5659)) - Improved streaming performance for tool-call arguments to run in linear time ([#5712](#5712)) - Fixed durable-mask config resolution with ranked resolver behavior ([#5672](#5672)) - Skipped background sync in Apple Terminal ([#5666](#5666)) - Hid thread IDs when tracing is disabled ([#5692](#5692)) - Kept installed providers visible in `/auth` ([#5689](#5689)) - Preloaded the auth UI before notification handoff ([#5697](#5697)) - Updated and clarified UI copy across Auto mode, YOLO hints, classifier notices, `/tokens`, line-number toggles, review failures, onboarding Tavily cancellation, and OpenAI subscription login labels ([#5685](#5685), [#5694](#5694), [#5684](#5684), [#5687](#5687), [#5680](#5680), [#5688](#5688), [#5686](#5686), [#5691](#5691), [#5693](#5693)) - Removed the `Muse Spark 1.1` recommendation ([#5683](#5683)) _End release notes preview._ --- > [!NOTE] > A **community contributors** list and a **Special thanks** section (crediting the users who filed the issues this release's PRs closed) are appended to the GitHub release notes automatically at publish time (see [Release Pipeline](https://github.com/langchain-ai/deepagents/blob/main/.github/RELEASING.md#release-pipeline), step 3). --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: langchain-oss-automated-triage[bot] <248757908+langchain-oss-automated-triage[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A
multi_selectanswer fromask_useris now a JSON array of the selected values (["a", "b"], or[]for an optional question left untouched) instead of the values joined with", ". Choice values and custom Other text may now contain commas, quotes, and newlines — the previous comma ban is gone. Invalidask_userpayloads (e.g. an emptyquestionslist or a blank choice value) now return a recoverable tool error the model can read and retry against, instead of crashing the turn with a fatalAgent error.The joined-string encoding existed only for transcript legibility — nothing ever parsed the joined string back into a list — but it forced
_validate_choicesto reject any value containing a comma, and the TUI to block Other text containing one. Worse, that rejection raisedValueErrorat the top of the tool body, whichToolNodedoes not convert into a tool-error message; it re-raised up the graph and killed the turn before the model ever saw what was wrong.The JSON-in-string encoding keeps the
answers: list[str]wire shape (one string per question, positionally matched), soAskUserAnswered, the authorization receipt, and_parse_answers' length/positional checks are untouched. JSON is self-delimiting and exactly invertible via the newencode_multi_select_answer/decode_multi_select_answerhelpers, so punctuation in a choice value is safe and malformed input can be rejected loudly rather than silently mis-split.Validation failures now raise
ToolException, and theask_usertool setshandle_tool_error = TruesoBaseTool.runconverts it into an errorToolMessage(the same patternmcp_toolsuses). Without that flag theToolExceptionwould still re-raise past langgraph's default error handling, which only mapsToolInvocationError. Error messages are unchanged in content — they still name the offending value and the exact problem — so a corrected retry normally succeeds in one attempt.Two subtleties worth careful review:
not get_answer().strip(). An untouched multi-select now encodes as the truthy string"[]", which would wrongly pass a required question; the check now goes through a decodinganswer_is_empty()helper.A:line. It is always a single line, so the blank-line-separatedQ:/A:layout is unchanged;textandmultiple_choiceanswers stay raw single strings.Test plan
make lint(ruff + ty) andmake formatclean.None; single-line encoding; transcript rendering the JSON form; an empty[]answer keeping its authorization receipt; Other text containing a comma submitting verbatim; an empty[]still failing a required question; and validation failures raisingToolException.libs/codeunit suite passes (one pre-existing, unrelatedtest_server_graph.pyfailure from ambient localauto_classifier_modelconfig leaking into a mock assertion).ToolNode: a comma-laden choice (push-to-main — no PR label, always strict) asks, submits, and returnsA: ["push-to-main — no PR label, always strict"]withstatus="success"; an invalid payload returnsstatus="error"with the actionable message and the turn continues.