Skip to content

feat(code): classifier-backed Auto approval mode - #4804

Merged
Mason Daugherty (mdrxy) merged 13 commits into
mainfrom
mdrxy/code/classifier-auto-mode
Jul 17, 2026
Merged

feat(code): classifier-backed Auto approval mode#4804
Mason Daugherty (mdrxy) merged 13 commits into
mainfrom
mdrxy/code/classifier-auto-mode

Conversation

@mdrxy

@mdrxy Mason Daugherty (mdrxy) commented Jul 16, 2026

Copy link
Copy Markdown
Member

Deep Agents Code now offers an opt-in Auto approval mode. Auto immediately runs a narrow, deterministic set of routine actions. For other approval-gated actions, the active model checks whether they match the user's literal request. It runs approved actions and blocks the rest. After repeated denials or classifier failures, Auto sends the next batch to the normal approval UI, then continues in Auto mode.

Auto is experimental and currently limited to interactive, local TUI sessions using FilesystemBackend. Manual remains the default.

Warning

Auto is an authorization heuristic for a local coding agent. It is not sandbox containment, an operating-system boundary, or a guarantee that model-generated actions are safe.

Enable Auto

  1. Set DEEPAGENTS_CODE_EXPERIMENTAL=1.
  2. Select Auto in one of these ways:
    • In-app: use Shift+Tab or Ctrl+T to toggle between Manual and Auto.
    • At launch: pass -y or --auto-approve.
    • By default: set [startup].mode = "auto" in ~/.deepagents/config.toml.

-y and --auto-approve now select Auto rather than unrestricted execution. Users who want the previous unrestricted behavior must choose --yolo. The removed dangerously-auto startup value is no longer recognized and falls back to Manual.


Why

The previous approval modes forced a binary choice:

  • Manual preserved a human checkpoint, but repeatedly interrupted routine coding work.
  • YOLO removed that checkpoint entirely, including for actions that exceeded the user's request (blanket approval).

Auto introduces a third policy: keep the low-friction path for clearly routine work, but make authorization explicit for potentially consequential actions.

Mode Policy Entry points Availability
Manual Uses the normal human approval UI for gated actions Default; [startup].mode = "manual" All interactive sessions
Auto Uses narrow deterministic rules, then classifier review, with human fallback after repeated denials or failures -y, --auto-approve, [startup].mode = "auto", or the live keyboard toggle Experimental, local, unsandboxed TUI sessions
YOLO Runs gated actions without review --yolo or [startup].mode = "yolo" Interactive sessions after a one-time acknowledgement

How Auto handles a proposed action

flowchart TD
    A[Model proposes tool calls] --> B{Covered by the approval policy?}
    B -->|No| X[Existing tool behavior is unchanged]
    B -->|Yes| S{Is this thread still in Auto, with readable denial/failure history?}
    S -->|No| J[Open the existing approval HITL widget]
    S -->|Yes| C{Narrow deterministic allow?}
    C -->|Yes| R[Execute without a classifier call]
    C -->|No| D[Build one structured decision batch]
    D --> H{Was this batch already reviewed, or do earlier denials/failures require human review?}
    H -->|Yes| J
    H -->|No| E[Active model reviews effects against user request/input]
    E -->|Allow| R
    E -->|Deny| Q{Total-denial threshold reached?}
    Q -->|No| F[Return a sanitized error result]
    Q -->|Yes| J
    E -->|Unavailable or invalid| G[Return a compact unavailable result]
    F --> I[Agent can revise its plan]
    G --> I
    J -->|Approve| R
    J -->|Reject| K[Return a rejection result]
    J -->|Switch to Manual| L[Persist Manual, then review the full gated batch]
    R --> M[Reconcile the result and continue]
    K --> M
    I --> M
    L --> M
Loading

In practice:

  1. Auto starts with Manual's existing human-in-the-loop (HITL) rules. It changes only how HITL-gated actions are reviewed.
    • These built-in tools are not HITL-gated: ls, read_file, glob, grep, ask_user, get_goal, get_rubric, and update_goal.
    • MCP tools with valid read-only annotations, the optional js_eval tool, and caller-supplied tools without HITL configuration continue to run normally.
  2. Only narrow cases resolve locally. The deterministic policy can allow, never deny:
    • A routine write such as src/parser.py can proceed based on its resolved path and suffix; a sensitive target such as .github/workflows/ci.yml goes to classifier review.

    • A read-only Git command such as git status can proceed; a mutating command such as git commit -m change goes to classifier review. Explicit shell allow-list entries still apply after Auto removes broad and wildcard rules.

    • An MCP tool with a valid read-only annotation can proceed; tools without read-only annotations, or with contradictory or malformed annotations, go to classifier review.

      How MCP read-only annotations are evaluated

      MCP servers can attach standard ToolAnnotations when advertising a tool. Deep Agents Code copies those annotations into the wrapped tool's metadata and lets the tool bypass classifier review only when:

      • readOnlyHint is the literal Boolean true;
      • destructiveHint is absent, null, or false; and
      • every supplied standard hint (readOnlyHint, destructiveHint, idempotentHint, or openWorldHint) is a Boolean or null, rather than a string or another type.

      For example:

      • readOnlyHint: true, destructiveHint: false proceeds without classifier review.
      • No readOnlyHint goes to review because the tool never explicitly claimed to be read-only.
      • readOnlyHint: true, destructiveHint: true goes to review because the hints contradict each other.
      • readOnlyHint: true, destructiveHint: "false" goes to review because "false" is a string, not a Boolean.

      In Auto, tools that do not pass this check enter the classifier batch. In Manual, they use the normal approval UI; headless runtimes reject them because they have no approval UI. The annotation is a server-provided assertion that Deep Agents Code does not independently verify.

  3. The classifier reviews all unresolved calls together. Only pre-expansion literal user text can authorize an action. Referenced paths, summarized arguments, prior tool calls, and MCP metadata provide effect context but cannot grant permission. Expanded files, assistant prose, and tool results are excluded.
    • Example: For the literal request Research issue #123, update @src/parser.py, and open a PR, proposed fetch_url and git push calls can be reviewed in one batch and receive separate decisions. The referenced path and summarized call arguments help identify what each action would affect, but the expanded file contents are not sent. Instructions found in that file, assistant prose, or a prior tool result therefore cannot widen the user's request or authorize another action.
  4. Auto revalidates before side effects. The decision plan is bound to the thread, mode, batch, and exact gated calls. Missing or invalid state, a mode race, or a replay falls back to human review. Denied calls are skipped and receive sanitized error results so the agent can revise its plan.
    • Examples: If the user switches to Manual while classifier review is in progress, an earlier Auto decision cannot execute silently; the normal approval UI opens instead. A plan associated with another thread, a different set of tool-call IDs, missing decisions, or a previously processed batch also requires human review. If Auto denies a proposed delete, it returns an error result with that call's ID instead of invoking the tool, allowing the agent to propose a safer next step.

Limits and user-visible behavior

  • The Manual approval menu can enable Auto for the current thread. Threshold fallback can switch permanently to Manual or perform a one-off review while leaving Auto enabled.
  • Denials, classifier failures, and fallbacks appear as short sanitized transcript events rather than raw classifier output.
  • Auto requested without the experimental opt-in, or in a sandboxed session, falls back to Manual with a warning. Explicit autonomous-mode flags are rejected in non-interactive and ACP entry points.
  • The active model is not an independent security authority. MCP read-only annotations are trusted as a deliberate beta tradeoff.
  • Parent-level Auto review does not cover actions performed inside delegated subagents or broader explicitly configured PTC/js_eval fan-out. Model providers and tracing backends may still observe classifier inputs and outputs even though the TUI hides them.
Implementation and reviewer details

Approval boundary and decision inputs

The following tools are HITL-gated:

  • execute, write_file, edit_file, and delete;
  • web_search and fetch_url;
  • task;
  • start_async_task, update_async_task, and cancel_async_task;
  • compact_conversation when optional compaction approval is enabled; and
  • MCP tools without valid read-only annotations.

For deterministic writes and edits, the fast path uses only the resolved target path and suffix; it does not inspect proposed content. Dependency and lock files, sensitive names and locations, shell scripts, unsupported file types, and paths outside the trusted root do not qualify. Selected Git inspection commands must also pass shell-control and outside-worktree path checks. Configured shell entries remain an explicit user trust exception, but broad commands and wildcard entries are ignored by Auto.

The classifier receives one typed batch containing:

  • up to 20 client-attached, pre-expansion prompt rows, including literal prompts, referenced path names, and turn IDs;
  • trusted project-root and redacted-origin facts; and
  • depth- and length-limited effect context from current arguments, prior AI tool calls after the latest trusted prompt, and selected MCP annotations.

Only literal_user_text is authorizing. Content-bearing write/edit arguments are reduced to character counts. The classifier must return exactly one unique decision for every unresolved tool-call ID; missing, duplicated, unknown, or malformed decisions make the classifier unavailable rather than partially authorizing the batch.

Before routing calls, Auto re-reads the live mode and validates its private decision plan. Manual mode, invalid control state, mismatched calls, or replayed batches require human review. Classifier review uses a second call to the active session model with a 20-second timeout. Raw classifier text and tool-call chunks are hidden from the transcript, while usage accounting and tracing are retained.

Denial reasons are sanitized before persistence or display by removing control characters, known credential values, credential-like assignments, and URL credentials or query values. MCP tools without valid read-only annotations are HITL-gated; headless runtimes reject them because no approval UI is available.

Failure and fallback behavior

Condition Result
Selected mode cannot be persisted Persist Manual instead; if that also fails, block graph execution
Classifier timeout, provider error, or invalid response Block the affected calls and return a compact unavailable result
Three consecutive policy denials in one user turn Send the next classifier-review-eligible calls to one-off human review
Two consecutive classifier-unavailable results Send the next classifier-review-eligible calls to one-off human review
A denial raises the thread's total denial count to 20 Require human review for that call immediately
Counter state cannot be read or reset before classification Require human review for every gated call in the batch
Counter state cannot be written after classification Require human review for classifier-path decisions; deterministic fast-path allows remain eligible
Decision plan is missing, malformed, replayed, or bound to another thread Require Manual review
User chooses Switch to Manual during fallback Persist Manual before presenting the full gated batch for review

Python API and state compatibility

  • Existing Boolean compatibility input retains its previous meaning: auto_approve=True maps to YOLO, not Auto.
  • Per-thread Store records persist the typed mode instead of a Boolean. Missing, legacy, or malformed records fail closed.
  • Startup and live mode changes take effect only after the Store write succeeds.
  • YOLO requires a versioned local acknowledgement and cannot be entered through the live keyboard toggle.

Reviewer guide

Suggested review order:

  1. Mode semantics and persistenceApprovalMode, Store payloads, acknowledgement, startup resolution, and live mode writes.
  2. Authorization policyAutoModeHITLMiddleware, literal prompt metadata, deterministic routing, structured classification, sanitization, counters, and checkpoint validation.
  3. Graph boundary — middleware replacement, the gated-tool inventory, MCP annotation handling, headless rejection, PTC, and delegated-subagent scope.
  4. Client behavior — stream metadata, classifier-output filtering, fallback interrupts, keyboard toggles, status presentation, and transcript events.

Highest-risk invariants:

  • every consequential tool in the supported boundary is gated or explicitly outside scope;
  • path-based writes and selected Git fast paths stay within their documented checks;
  • only literal user prompts can authorize classifier-reviewed effects;
  • malformed output, Store failures, mode races, and checkpoint replay fail closed; and
  • hidden classifier chunks never create transcript content while usage accounting remains intact.

@github-actions github-actions Bot added dcode Related to `deepagents-code` feature New feature/enhancement or request for one internal User is a member of the `langchain-ai` GitHub organization size: XL 1000+ LOC labels Jul 16, 2026

@open-swe open-swe Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Open SWE Review found 1 potential issue.

Open in WebView Open SWE trace

Comment thread libs/code/deepagents_code/auto_mode.py Outdated

@corridor-security corridor-security Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Auto mode's _fixed_repo_command_allowed regex omits the single & shell control operator, allowing an attacker-influenced execute tool call like pytest & rm -rf . to bypass deterministic approval and run an arbitrary background command without human review.

Comment thread libs/code/deepagents_code/auto_mode.py Outdated

@corridor-security corridor-security Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new deterministic allow logic in Auto mode unconditionally bypasses classifier review for build/test/package-manager commands (make, npm/yarn/pnpm, pytest, cargo, go, uv run) that execute arbitrary project-defined code, creating an unreviewed code-execution path for users working against a malicious or compromised repository when Auto is enabled.

Comment thread libs/code/deepagents_code/auto_mode.py
@mdrxy Mason Daugherty (mdrxy) changed the title feat(code): add classifier-backed Auto approval mode feat(code): classifier-backed Auto approval mode Jul 17, 2026
@mdrxy
Mason Daugherty (mdrxy) merged commit eae7c28 into main Jul 17, 2026
93 checks passed
@mdrxy
Mason Daugherty (mdrxy) deleted the mdrxy/code/classifier-auto-mode branch July 17, 2026 21:09
Mason Daugherty (mdrxy) pushed a commit that referenced this pull request Jul 17, 2026
> [!CAUTION]
> Merging this PR will automatically publish to **PyPI** and create a
**GitHub release**.

For the full release process, see
[`.github/RELEASING.md`](https://github.com/langchain-ai/deepagents/blob/main/.github/RELEASING.md).

---

_Release notes preview: keep this section in sync with the package
`CHANGELOG.md`. The published GitHub release body is extracted from the
merged `CHANGELOG.md` by `release.yml`, not from this PR description._

---


##
[0.1.43](deepagents-code==0.1.42...deepagents-code==0.1.43)
(2026-07-17)

### Features

- Added classifier-backed Auto approval mode behind
`DEEPAGENTS_CODE_EXPERIMENTAL=1`
([#4804](#4804)).
- Added a shutdown toast for deferred exits
([#4830](#4830)).
- Task descriptions that were truncated can now be expanded by clicking
or pressing `Ctrl+O`
([#4811](#4811)).
- Debug Console clears with `Ctrl+L` now persist after reopening
([#4812](#4812)).
- Added debug logging for skill-name override collisions
([#4772](#4772)).

### Bug Fixes

- Keep chat input responsive during `/restart`
([#4808](#4808)).
- Fixed paste placeholders disappearing when backspacing a newline below
them ([#4757](#4757)).
- Made markdown `AppMessage` output selectable
([#4814](#4814)).
- Fixed live tool-group counts to include only running tools
([#4809](#4809)).
- Kept `task` timers monotonic across nested subagent human-in-the-loop
flows ([#4771](#4771)).
- Preserved goal criteria proposals when marker clearing fails
([#4785](#4785)).
- Reduced repeated probing of an unreachable Ollama daemon to once per
reload
([#4806](#4806)).
- Quieted MCP auth-skip debug logging for known patterns
([#4805](#4805)).
- Improved `/version` diagnostics for editable installs and core
dependency reporting, including surfacing `langchain-quickjs`
([#4816](#4816),
[#4813](#4813)).
- Removed the `uv install` tip from the `/version` update hint
([#4822](#4822)).

_End release notes preview._

---

> [!NOTE]
> A **New Contributors** section is appended to the GitHub release notes
automatically at publish time (see [Release
Pipeline](https://github.com/langchain-ai/deepagents/blob/main/.github/RELEASING.md#release-pipeline),
step 2).

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: langchain-oss-automated-triage[bot] <248757908+langchain-oss-automated-triage[bot]@users.noreply.github.com>
Mason Daugherty (mdrxy) added a commit to langchain-ai/docs that referenced this pull request Jul 21, 2026
…4993)

Documents the Manual/Auto/YOLO approval modes shipped in
`deepagents-code` 0.1.42
([langchain-ai/deepagents#4804](langchain-ai/deepagents#4804)).
Marcelo5444 pushed a commit to Marcelo5444/deepagents that referenced this pull request Jul 30, 2026
Deep Agents Code now offers an opt-in **Auto** approval mode. Auto
immediately runs a narrow, deterministic set of routine actions. For
other approval-gated actions, the active model checks whether they match
the user's literal request. It runs approved actions and blocks the
rest. After repeated denials or classifier failures, Auto sends the next
batch to the normal approval UI, then continues in Auto mode.

Auto is experimental and currently limited to interactive, local TUI
sessions using `FilesystemBackend`. Manual remains the default.

> [!WARNING]
> Auto is an authorization heuristic for a local coding agent. It is
**not** sandbox containment, an operating-system boundary, or a
guarantee that model-generated actions are safe.

## Enable Auto

1. Set `DEEPAGENTS_CODE_EXPERIMENTAL=1`.
2. Select Auto in one of these ways:
- In-app: use <kbd>Shift</kbd>+<kbd>Tab</kbd> or
<kbd>Ctrl</kbd>+<kbd>T</kbd> to toggle between Manual and Auto.
   - At launch: pass `-y` or `--auto-approve`.
- By default: set `[startup].mode = "auto"` in
`~/.deepagents/config.toml`.

`-y` and `--auto-approve` now select Auto rather than unrestricted
execution. Users who want the previous unrestricted behavior must choose
`--yolo`. The removed `dangerously-auto` startup value is no longer
recognized and falls back to Manual.

---

## Why

The previous approval modes forced a binary choice:

- **Manual** preserved a human checkpoint, but repeatedly interrupted
routine coding work.
- **YOLO** removed that checkpoint entirely, including for actions that
exceeded the user's request (blanket approval).

Auto introduces a third policy: keep the low-friction path for clearly
routine work, but make authorization explicit for potentially
consequential actions.

| Mode | Policy | Entry points | Availability |
|---|---|---|---|
| **Manual** | Uses the normal human approval UI for gated actions |
Default; `[startup].mode = "manual"` | All interactive sessions |
| **Auto** | Uses narrow deterministic rules, then classifier review,
with human fallback after repeated denials or failures | `-y`,
`--auto-approve`, `[startup].mode = "auto"`, or the live keyboard toggle
| Experimental, local, unsandboxed TUI sessions |
| **YOLO** | Runs gated actions without review | `--yolo` or
`[startup].mode = "yolo"` | Interactive sessions after a one-time
acknowledgement |

## How Auto handles a proposed action

```mermaid
flowchart TD
    A[Model proposes tool calls] --> B{Covered by the approval policy?}
    B -->|No| X[Existing tool behavior is unchanged]
    B -->|Yes| S{Is this thread still in Auto, with readable denial/failure history?}
    S -->|No| J[Open the existing approval HITL widget]
    S -->|Yes| C{Narrow deterministic allow?}
    C -->|Yes| R[Execute without a classifier call]
    C -->|No| D[Build one structured decision batch]
    D --> H{Was this batch already reviewed, or do earlier denials/failures require human review?}
    H -->|Yes| J
    H -->|No| E[Active model reviews effects against user request/input]
    E -->|Allow| R
    E -->|Deny| Q{Total-denial threshold reached?}
    Q -->|No| F[Return a sanitized error result]
    Q -->|Yes| J
    E -->|Unavailable or invalid| G[Return a compact unavailable result]
    F --> I[Agent can revise its plan]
    G --> I
    J -->|Approve| R
    J -->|Reject| K[Return a rejection result]
    J -->|Switch to Manual| L[Persist Manual, then review the full gated batch]
    R --> M[Reconcile the result and continue]
    K --> M
    I --> M
    L --> M
```

In practice:

1. **Auto starts with Manual's existing human-in-the-loop (HITL)
rules.** It changes only how HITL-gated actions are reviewed.
- These built-in tools are not HITL-gated: `ls`, `read_file`, `glob`,
`grep`, `ask_user`, `get_goal`, `get_rubric`, and `update_goal`.
- MCP tools with valid read-only annotations, the optional `js_eval`
tool, and caller-supplied tools without HITL configuration continue to
run normally.
2. **Only narrow cases resolve locally.** The deterministic policy can
allow, never deny:
- A routine write such as `src/parser.py` can proceed based on its
resolved path and suffix; a sensitive target such as
`.github/workflows/ci.yml` goes to classifier review.
- A read-only Git command such as `git status` can proceed; a mutating
command such as `git commit -m change` goes to classifier review.
Explicit shell allow-list entries still apply after Auto removes broad
and wildcard rules.
- An MCP tool with a valid read-only annotation can proceed; tools
without read-only annotations, or with contradictory or malformed
annotations, go to classifier review.

     <details>
     <summary>How MCP read-only annotations are evaluated</summary>

MCP servers can attach standard `ToolAnnotations` when advertising a
tool. Deep Agents Code copies those annotations into the wrapped tool's
metadata and lets the tool bypass classifier review only when:

     - `readOnlyHint` is the literal Boolean `true`;
     - `destructiveHint` is absent, `null`, or `false`; and
- every supplied standard hint (`readOnlyHint`, `destructiveHint`,
`idempotentHint`, or `openWorldHint`) is a Boolean or `null`, rather
than a string or another type.

     For example:

- `readOnlyHint: true, destructiveHint: false` proceeds without
classifier review.
- No `readOnlyHint` goes to review because the tool never explicitly
claimed to be read-only.
- `readOnlyHint: true, destructiveHint: true` goes to review because the
hints contradict each other.
- `readOnlyHint: true, destructiveHint: "false"` goes to review because
`"false"` is a string, not a Boolean.

In Auto, tools that do not pass this check enter the classifier batch.
In Manual, they use the normal approval UI; headless runtimes reject
them because they have no approval UI. The annotation is a
server-provided assertion that Deep Agents Code does not independently
verify.

     </details>
3. **The classifier reviews all unresolved calls together.** Only
pre-expansion literal user text can authorize an action. Referenced
paths, summarized arguments, prior tool calls, and MCP metadata provide
effect context but cannot grant permission. Expanded files, assistant
prose, and tool results are excluded.
- **Example:** For the literal request `Research issue langchain-ai#123, update
@src/parser.py, and open a PR`, proposed `fetch_url` and `git push`
calls can be reviewed in one batch and receive separate decisions. The
referenced path and summarized call arguments help identify what each
action would affect, but the expanded file contents are not sent.
Instructions found in that file, assistant prose, or a prior tool result
therefore cannot widen the user's request or authorize another action.
4. **Auto revalidates before side effects.** The decision plan is bound
to the thread, mode, batch, and exact gated calls. Missing or invalid
state, a mode race, or a replay falls back to human review. Denied calls
are skipped and receive sanitized error results so the agent can revise
its plan.
- **Examples:** If the user switches to Manual while classifier review
is in progress, an earlier Auto decision cannot execute silently; the
normal approval UI opens instead. A plan associated with another thread,
a different set of tool-call IDs, missing decisions, or a previously
processed batch also requires human review. If Auto denies a proposed
`delete`, it returns an error result with that call's ID instead of
invoking the tool, allowing the agent to propose a safer next step.

## Limits and user-visible behavior

- The Manual approval menu can enable Auto for the current thread.
Threshold fallback can switch permanently to Manual or perform a one-off
review while leaving Auto enabled.
- Denials, classifier failures, and fallbacks appear as short sanitized
transcript events rather than raw classifier output.
- Auto requested without the experimental opt-in, or in a sandboxed
session, falls back to Manual with a warning. Explicit autonomous-mode
flags are rejected in non-interactive and ACP entry points.
- The active model is not an independent security authority. MCP
read-only annotations are trusted as a deliberate beta tradeoff.
- Parent-level Auto review does not cover actions performed inside
delegated subagents or broader explicitly configured PTC/`js_eval`
fan-out. Model providers and tracing backends may still observe
classifier inputs and outputs even though the TUI hides them.

<details>
<summary><strong>Implementation and reviewer details</strong></summary>

### Approval boundary and decision inputs

The following tools are HITL-gated:

- `execute`, `write_file`, `edit_file`, and `delete`;
- `web_search` and `fetch_url`;
- `task`;
- `start_async_task`, `update_async_task`, and `cancel_async_task`;
- `compact_conversation` when optional compaction approval is enabled;
and
- MCP tools without valid read-only annotations.

For deterministic writes and edits, the fast path uses only the resolved
target path and suffix; it does not inspect proposed content. Dependency
and lock files, sensitive names and locations, shell scripts,
unsupported file types, and paths outside the trusted root do not
qualify. Selected Git inspection commands must also pass shell-control
and outside-worktree path checks. Configured shell entries remain an
explicit user trust exception, but broad commands and wildcard entries
are ignored by Auto.

The classifier receives one typed batch containing:

- up to 20 client-attached, pre-expansion prompt rows, including literal
prompts, referenced path names, and turn IDs;
- trusted project-root and redacted-origin facts; and
- depth- and length-limited effect context from current arguments, prior
AI tool calls after the latest trusted prompt, and selected MCP
annotations.

Only `literal_user_text` is authorizing. Content-bearing write/edit
arguments are reduced to character counts. The classifier must return
exactly one unique decision for every unresolved tool-call ID; missing,
duplicated, unknown, or malformed decisions make the classifier
unavailable rather than partially authorizing the batch.

Before routing calls, Auto re-reads the live mode and validates its
private decision plan. Manual mode, invalid control state, mismatched
calls, or replayed batches require human review. Classifier review uses
a second call to the active session model with a 20-second timeout. Raw
classifier text and tool-call chunks are hidden from the transcript,
while usage accounting and tracing are retained.

Denial reasons are sanitized before persistence or display by removing
control characters, known credential values, credential-like
assignments, and URL credentials or query values. MCP tools without
valid read-only annotations are HITL-gated; headless runtimes reject
them because no approval UI is available.

### Failure and fallback behavior

| Condition | Result |
|---|---|
| Selected mode cannot be persisted | Persist Manual instead; if that
also fails, block graph execution |
| Classifier timeout, provider error, or invalid response | Block the
affected calls and return a compact unavailable result |
| Three consecutive policy denials in one user turn | Send the next
classifier-review-eligible calls to one-off human review |
| Two consecutive classifier-unavailable results | Send the next
classifier-review-eligible calls to one-off human review |
| A denial raises the thread's total denial count to 20 | Require human
review for that call immediately |
| Counter state cannot be read or reset before classification | Require
human review for every gated call in the batch |
| Counter state cannot be written after classification | Require human
review for classifier-path decisions; deterministic fast-path allows
remain eligible |
| Decision plan is missing, malformed, replayed, or bound to another
thread | Require Manual review |
| User chooses **Switch to Manual** during fallback | Persist Manual
before presenting the full gated batch for review |

### Python API and state compatibility

- Existing Boolean compatibility input retains its previous meaning:
`auto_approve=True` maps to YOLO, not Auto.
- Per-thread Store records persist the typed mode instead of a Boolean.
Missing, legacy, or malformed records fail closed.
- Startup and live mode changes take effect only after the Store write
succeeds.
- YOLO requires a versioned local acknowledgement and cannot be entered
through the live keyboard toggle.

### Reviewer guide

Suggested review order:

1. **Mode semantics and persistence** — `ApprovalMode`, Store payloads,
acknowledgement, startup resolution, and live mode writes.
2. **Authorization policy** — `AutoModeHITLMiddleware`, literal prompt
metadata, deterministic routing, structured classification,
sanitization, counters, and checkpoint validation.
3. **Graph boundary** — middleware replacement, the gated-tool
inventory, MCP annotation handling, headless rejection, PTC, and
delegated-subagent scope.
4. **Client behavior** — stream metadata, classifier-output filtering,
fallback interrupts, keyboard toggles, status presentation, and
transcript events.

Highest-risk invariants:

- every consequential tool in the supported boundary is gated or
explicitly outside scope;
- path-based writes and selected Git fast paths stay within their
documented checks;
- only literal user prompts can authorize classifier-reviewed effects;
- malformed output, Store failures, mode races, and checkpoint replay
fail closed; and
- hidden classifier chunks never create transcript content while usage
accounting remains intact.

</details>
Marcelo5444 pushed a commit to Marcelo5444/deepagents that referenced this pull request Jul 30, 2026
> [!CAUTION]
> Merging this PR will automatically publish to **PyPI** and create a
**GitHub release**.

For the full release process, see
[`.github/RELEASING.md`](https://github.com/langchain-ai/deepagents/blob/main/.github/RELEASING.md).

---

_Release notes preview: keep this section in sync with the package
`CHANGELOG.md`. The published GitHub release body is extracted from the
merged `CHANGELOG.md` by `release.yml`, not from this PR description._

---


##
[0.1.43](langchain-ai/deepagents@deepagents-code==0.1.42...deepagents-code==0.1.43)
(2026-07-17)

### Features

- Added classifier-backed Auto approval mode behind
`DEEPAGENTS_CODE_EXPERIMENTAL=1`
([langchain-ai#4804](langchain-ai#4804)).
- Added a shutdown toast for deferred exits
([langchain-ai#4830](langchain-ai#4830)).
- Task descriptions that were truncated can now be expanded by clicking
or pressing `Ctrl+O`
([langchain-ai#4811](langchain-ai#4811)).
- Debug Console clears with `Ctrl+L` now persist after reopening
([langchain-ai#4812](langchain-ai#4812)).
- Added debug logging for skill-name override collisions
([langchain-ai#4772](langchain-ai#4772)).

### Bug Fixes

- Keep chat input responsive during `/restart`
([langchain-ai#4808](langchain-ai#4808)).
- Fixed paste placeholders disappearing when backspacing a newline below
them ([langchain-ai#4757](langchain-ai#4757)).
- Made markdown `AppMessage` output selectable
([langchain-ai#4814](langchain-ai#4814)).
- Fixed live tool-group counts to include only running tools
([langchain-ai#4809](langchain-ai#4809)).
- Kept `task` timers monotonic across nested subagent human-in-the-loop
flows ([langchain-ai#4771](langchain-ai#4771)).
- Preserved goal criteria proposals when marker clearing fails
([langchain-ai#4785](langchain-ai#4785)).
- Reduced repeated probing of an unreachable Ollama daemon to once per
reload
([langchain-ai#4806](langchain-ai#4806)).
- Quieted MCP auth-skip debug logging for known patterns
([langchain-ai#4805](langchain-ai#4805)).
- Improved `/version` diagnostics for editable installs and core
dependency reporting, including surfacing `langchain-quickjs`
([langchain-ai#4816](langchain-ai#4816),
[langchain-ai#4813](langchain-ai#4813)).
- Removed the `uv install` tip from the `/version` update hint
([langchain-ai#4822](langchain-ai#4822)).

_End release notes preview._

---

> [!NOTE]
> A **New Contributors** section is appended to the GitHub release notes
automatically at publish time (see [Release
Pipeline](https://github.com/langchain-ai/deepagents/blob/main/.github/RELEASING.md#release-pipeline),
step 2).

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: langchain-oss-automated-triage[bot] <248757908+langchain-oss-automated-triage[bot]@users.noreply.github.com>
Mason Daugherty (mdrxy) added a commit that referenced this pull request Jul 31, 2026
Closes #4519

Deep Agents Code now keeps a running estimated USD total for the active
thread. The status bar shows it beside token usage, `/cost` and
`/tokens` break it down on demand, and usage summaries include per-model
cost. The total covers everything the thread spends — assistant turns,
subagents, offload/compaction, and Auto mode's classifier — and it is
restored on resume and thread switches.

---

## Where the number lives

A thread's lifetime cost should be durable to support `--resume` and
switching between threads. Consequently, the graph owns it and the
client (e.g. TUI / headless CLI) can read it.

Concretely, `_session_cost_usd` is a new channel on the agent's state
schema (`CostState`, which extends `ResumeState`), which means it is
persisted by the graph's checkpointer, in the same checkpoint as
`messages`.

> <details>
> <summary>FAQ: <code>CostState</code>, and why it extends
<code>ResumeState</code></summary>
>
> `CostState` is the [state
schema](https://docs.langchain.com/oss/python/langgraph/graph-api#schema)
`CostTrackingMiddleware` declares — the agent's state plus one channel:
>
> ```python
> class CostState(ResumeState):
> _session_cost_usd: Annotated[NotRequired[float], PrivateStateAttr,
operator.add]
> ```
>
> **What `ResumeState` already holds.** It's the schema for facts that
belong to the thread as a whole rather than to any one message — the
things the TUI/client needs the moment you open a thread, before the
agent runs again:
>
> - `_context_tokens` — how full the context window was after the last
turn. This drives token readout — e.g., the display in the TUI's footer
status bar, where the cost figure now also sits.
> - `_model_spec` / `_model_params` — the model and settings the thread
was using, so `-r` resumes on _that_ model instead of whatever your
current [global
default](https://docs.langchain.com/oss/deepagents/code/config-file#default-and-recent-model)
might happen to be.
> - `_goal_objective`, `_goal_rubric`, `_goal_status`,
`_goal_status_note`, `_sticky_rubric`, and the pending-proposal fields —
the
[goal](https://docs.langchain.com/oss/deepagents/code/goals-and-rubrics)
the thread is working toward (if one is set) and where progress on said
goal stands.
>
> **Every field here is private, including the new one.** A graph has a
public interface: its input and output schemas say what callers may pass
in and receive back (`messages` is the obvious example). None of the
fields above belong in that interface, as they aren't things a caller
supplies or asks for — they're notes that dcode writes for itself so it
can pick the thread back up later.
>
> `PrivateStateAttr` omits a field from those public input/output
schemas. It does **not** make the field ephemeral or unreadable: the
channel remains ordinary checkpointed state, is available to graph
nodes, and is returned by `get_state`. Its writes can also appear as raw
deltas in the `updates` stream; the live status bar uses a dedicated
absolute-total event instead so the client does not have to reproduce
the reducer (details below).
>
> **These values are stored rather than recomputed**, because
recomputing them is expensive or impossible after the fact. Reopening a
thread has to show the context gauge and the current model immediately.
The context number originally came from the last response's
`usage_metadata` — the provider counted it for us. If it weren't saved,
the CLI would have to load the whole message history and count tokens
itself to guess at it: slow, dependent on having a tokenizer that
matches the provider's, and still only an approximation of what was
actually billed. Cost is the same shape of problem and worse —
re-deriving it would mean re-pricing every message in the thread every
time you open it.
>
> **So why extend `ResumeState`?** Cost is the same kind of resumable
thread fact, and pricing also needs the model information that
`ResumeState` already carries. LangChain's `AIMessage` contract does not
require a model identifier: `usage_metadata` standardizes token counts,
while `response_metadata` is optional. Provider integrations commonly
include a model name, but custom and third-party integrations are
allowed to omit it. When a response does, pricing falls back to the
`_model_spec` persisted for that turn.
>
> </details>

Riding the checkpoint is a deliberate choice over an alternative where
the client prices a turn and then pushes the new total with
`aupdate_state`. (That's a second round trip that can be interrupted —
e.g. `Ctrl-C`, a crash, a dropped connection — *after* the messages have
already been committed.) Writing from inside the graph puts the cost in
the same commit as the response that incurred it. Importantly, it also
keeps an `UpdateState` run/span out of LangSmith traces if configured.
Restoring is then a single `aget_state` read from the checkpoint.

Two things about the channel's declaration matter downstream:

- **Additive.** Its reducer is `operator.add`, so a writer returns *only
the delta it just priced* — literally `{"_session_cost_usd": 0.02}`,
never a replacement total. If a thread starts a turn at $1.00,
`after_model` contributes $0.02 and a later rubric-grading drain
contributes $0.005, the checkpoint ends at $1.025. Neither hook has to
perform its own read-modify-write.
- **Private.** It carries `PrivateStateAttr`, keeping cost out of the
graph's public input/output contract while leaving it checkpointed and
readable through `get_state`. Deep Agents also uses this marker to
identify state that must not cross the main-agent/subagent boundary.

The cumulative channel has one writer: the `CostTrackingMiddleware`
instance belonging to the thread's main agent. Subagents run the same
middleware, but theirs is constructed with `nested=True`, which switches
its pricing off — a subagent's spend is charged by its parent instead,
as the next section explains.

To get every completed model call to that writer, the cost-tracking
module installs one process-wide **Recorder**. The Recorder is an
in-memory inbox for completed model calls, implemented as a LangChain
callback (`BaseCallbackHandler`). When a request finishes, it files the
request's usage, model/provider metadata, and message ID under that
thread. It does not calculate cost or write graph state; it only holds
records until the main agent's middleware drains them.

```mermaid
flowchart LR
    subgraph calls["every model call in the run"]
        direction TB
        A["assistant"]
        B["subagent"]
        C["offload / summarization"]
        D["Auto mode classifier"]
        E["rubric grader"]
    end
    calls --> REC["in-memory model-call recorder<br/>collects usage, keyed by thread"]
    REC --> MW["CostTrackingMiddleware<br/>drains, prices, adds the delta"]
    MW --> CK[("checkpoint<br/>_session_cost_usd")]
    MW -. "stream event: absolute total" .-> BAR["status bar and /cost"]
    CK -. "resume, thread switch, end of turn" .-> BAR
```

The client is purely a reader; nothing outside the graph writes to the
channel.

## Why collecting is split from pricing

The obvious place to price a request is `after_model`, but that hook
only ever sees the agent's own model node, and a large share of a
thread's spend can happen somewhere else entirely:

| Model call | Runs in |
| --- | --- |
| Assistant turn, [Auto
mode](#4804) classifier |
model node |
| Offload /
[summarization](https://docs.langchain.com/oss/python/deepagents/context-engineering#summarization)
| `before_model` (stock) or the model wrap chain (deepagents) |
| Subagent turns | tool node, in a child graph |
| `compact_conversation` | tool node |
| [Rubric
grading](https://docs.langchain.com/oss/deepagents/code/goals-and-rubrics)
| `after_agent` |

The Recorder and the middleware have complementary access:

- The Recorder sits at LangChain's callback layer, so it can observe
model calls made by the main agent, a subagent, summarization, or
another direct invocation. But a callback is not a graph node and cannot
return an update to `_session_cost_usd`.
- `CostTrackingMiddleware` runs inside the main agent's graph, so its
hooks can return an additive update that the checkpointer persists. But
those hooks do not run inside a child graph or around a bare side
invocation, so they cannot discover all spend on their own.

The in-memory queue bridges those two scopes. The callback places one
record in the queue when a request completes; `CostTrackingMiddleware`
drains the current thread's records in `after_model` and `after_agent`,
prices them, and returns their sum as one state update.

> <details>
> <summary>Example: one turn with hidden model work</summary>
> 
> Suppose a thread already sits at $1.00. Before the main response,
summarization spends $0.004; the main response then spends $0.020. The
Recorder files both completions under the thread. `after_model` drains
those two records and writes `+0.024`, taking the checkpoint to $1.024.
> 
> If rubric grading then spends $0.005 after the final model step,
`after_agent` drains that later record and writes `+0.005`. The
committed total is $1.029. A subagent call works the same way: its
record waits in the shared thread queue until the parent's next drain.
> 
> </details>

> <details>
> <summary>Why the main response is also checked in state</summary>
> 
> Normally, a main-agent response with message ID `m1` produces a
Recorder entry for `m1`. The middleware prices that entry, remembers
that `m1` was charged, and skips the copy of `m1` already present in
graph state.
> 
> If a custom or third-party integration returns an `AIMessage` with
usage metadata but no usable callback record was filed — for example,
the callback could not associate the request with a thread —
`after_model` can still price that main response from state. Comparing
message IDs lets the normal and fallback paths coexist without
double-counting. This fallback is limited to the main response; hidden
calls are not in the parent's state and therefore still rely on the
Recorder.
> 
> </details>

## How subagent spend is counted

An earlier version of this PR had subagents accumulate into their own
copy of the channel and merge the total back. That silently did nothing
because Deep Agents intentionally treats `PrivateStateAttr` as
agent-local state: `SubAgentMiddleware` discovers those keys and strips
them both when seeding a child and when merging its result. This
prevents one agent's internal bookkeeping from leaking into another and
avoids parallel subagents racing to write private `LastValue` channels.
A parent-visible aggregate would need a separate shared/public contract;
for cost, the thread-keyed recorder supplies that bridge without
weakening private-state isolation.

Since a subagent inherits its parent's thread ID, its completed model
calls land in the same in-memory recorder queue. The parent drains and
prices those records on its next `after_model` or `after_agent` hook:

```mermaid
sequenceDiagram
    participant P as Main agent
    participant R as In-memory recorder
    participant S as Subagent
    participant C as Checkpoint
    P->>R: assistant call usage
    R-->>P: drain 1 record
    P->>C: add priced delta
    P->>S: task tool
    S->>R: nested call usage
    Note over S: Nested cost middleware does not<br/>price or accumulate spend
    S-->>P: result
    P->>R: next assistant call usage
    R-->>P: drain nested + assistant records
    P->>C: add priced delta
```

## Keeping the status bar live

The `updates` stream can expose the private channel's writes, but those
writes are reducer inputs such as `+0.02`, not the post-reducer lifetime
total. Making the client accumulate them would create a second copy of
the graph's state machine, vulnerable to missed stream chunks. Instead,
the writer emits a small `custom` event carrying the **absolute** total
and the client applies it with `set_cost`.

> <details>
> <summary>How an absolute event recovers from a dropped
update</summary>
> 
> Suppose the client shows $1.00. The graph commits another $0.02 and
emits `$1.02`, but that event is dropped. A later $0.005 charge emits
`$1.025`. Because the payload is an absolute value, the client replaces
$1.00 with $1.025 and is correct again; a delta-only stream would leave
it permanently short by the missed $0.02. The end-of-turn checkpoint
read provides another reconciliation point if the final event is the one
that disappears.
> 
> </details>

The provisional number exists only to make the status bar react before
the graph reaches its next cost-tracking hook. These two client-side
fields have different jobs:

- `_session_cost_usd` is the latest absolute total received from the
graph. It is the durable value restored from the checkpoint.
- `_provisional_cost_usd` is a temporary display estimate derived from
usage on the message stream. It is never written to graph state.

For a request estimated at $0.02, the display moves through this
sequence:

| Moment | Durable total | Provisional | Status bar |
| --- | ---: | ---: | ---: |
| Before the request | $1.00 | $0.00 | $1.00 |
| Usage arrives on the message stream | $1.00 | $0.02 | $1.02 |
| The graph emits its new absolute total | $1.02 | $0.00 | $1.02 |

The last step replaces the durable total and clears the provisional
bucket, so the same request is not shown twice. The displayed number
stays steady at $1.02; only its source changes from an early estimate to
checkpointed state. On a completed turn, the final checkpoint read
provides the same reconciliation if the custom event was missed.

`/cost`'s per-type and per-model rows remain client-side and stay
labeled “since this thread was loaded” — only the lifetime total above
them comes from the graph.

## Pricing

Pricing stays in one place: `estimate_cost` is the only function that
touches canonical price data, currently through `genai-prices`. For each
request it needs split input/output token counts, the model, and — when
the model name is available from more than one host — the provider.
Provider names are normalized to the identifiers in the price catalog;
if response metadata omits the model or provider, the checkpointed model
spec is the fallback.

Here are the main pricing paths.

**Ordinary input and output.** Suppose a catalog entry charges $2 per
million input tokens and $8 per million output tokens. A request with
10,000 input tokens and 2,000 output tokens is estimated as:

```text
input:  10,000 / 1,000,000 × $2 = $0.020
output:  2,000 / 1,000,000 × $8 = $0.016
total:                              $0.036
```

(Those rates are illustrative; `genai-prices` supplies the actual
model/provider rates.)

**Provider-specific lookup.** The response's provider matters when the
same model family is sold through different hosts. For example,
LangChain's `azure_openai` provider name is normalized to the catalog's
`azure` identifier, while a direct OpenAI request uses `openai`. This
prevents a recognizable model name from silently selecting the wrong
host's price.

**Cached input.** LangChain's `input_tokens` is inclusive of cache reads
and writes. If a response reports 1,000 input tokens with 600 cache-read
tokens and 100 cache-write tokens, only 300 are ordinary input:

```text
ordinary input: 1,000 - 600 - 100 = 300
cache reads:                            600
cache writes:                           100
```

`estimate_cost` passes all three buckets to `genai-prices`, which
applies the ordinary-input, cache-read, and cache-write rates
separately. Output tokens are priced independently. If malformed
provider metadata says the cache buckets exceed the inclusive input
count, the tracker clamps them to the input total rather than creating a
negative ordinary-input count.

**Anthropic cache TTLs.** When Anthropic reports its TTL breakdown,
LangChain sets generic `cache_creation` to zero and exposes
`ephemeral_5m_input_tokens` and `ephemeral_1h_input_tokens` instead. If
those fields contain 100 and 50 tokens, the tracker passes 150
cache-write tokens rather than accidentally billing them as ordinary
input. `genai-prices` currently exposes one cache-write bucket, so the
two TTL buckets share that catalog estimate.

The estimator declines to guess when the available data cannot support a
defensible calculation:

- A response with only `total_tokens=1_200` is unpriceable because those
tokens could be input, output, or a mixture, and the rates usually
differ.
- An unknown model has no canonical catalog rate to apply.
- Subscription-style access such as the Codex provider is not equivalent
to per-token API billing.
- Malformed or non-finite usage/price data is ignored rather than
allowed to fail the model turn.

In each case `estimate_cost` returns `None`; the request is omitted from
the dollar total while its token usage can still appear in `/tokens`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dcode Related to `deepagents-code` feature New feature/enhancement or request for one internal User is a member of the `langchain-ai` GitHub organization size: XL 1000+ LOC

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant