Skip to content

Avoid multimodal path allocation on text-only turns - #1789

Merged
xeophon merged 1 commit into
feat/nano-as-v1from
codex/v1-text-only-mm-path
Jun 21, 2026
Merged

Avoid multimodal path allocation on text-only turns#1789
xeophon merged 1 commit into
feat/nano-as-v1from
codex/v1-text-only-mm-path

Conversation

@xeophon

@xeophon xeophon commented Jun 21, 2026

Copy link
Copy Markdown
Member

Overview

Avoid materializing a full (node_id, message) path for text-only V1 turns. New node IDs are tracked independently for routed-expert attribution, while multimodal turns continue to build the complete prompt path needed for cursor alignment.

Why

_commit_turn previously built a tuple path for every reused prompt message before calling _attribute_mm. On text-only turns, multi_modal_data is absent and that helper returns immediately, leaving an O(P) container allocation and pass with no multimodal work to perform.

Training responses still perform the required O(P) token-prefix validation. This change removes the separate, redundant path construction; it does not claim to make training commits sublinear.

Implementation

  • Read the turn's multimodal sidecar once and create the full message path only when it is present.
  • Record new input and assistant node IDs as nodes are appended, preserving routed-expert attribution without depending on the multimodal structure.
  • Preserve full-prefix scanning for later multimodal turns so reused images advance modality cursors and new images attach to the correct nodes.

Performance

An isolated PEP 723 microbenchmark used 200,000 reused text messages, three repetitions, time.perf_counter() for median wall time, and tracemalloc for Python allocations.

Metric Previous New Saved
Median wall time 0.110177 s 0.046306 s 0.063871 s (58.0%)
Peak traced allocation 14,626,822 B 1,802,766 B 12,824,056 B (87.7%)
Live traced allocation after return 1,914,217 B 1,802,137 B 112,080 B

The benchmark isolates the removed allocation. End-to-end training gains may be smaller because token-prefix validation remains, while the peak temporary-memory reduction directly reflects the eliminated list and tuple containers.


Note

Low Risk
Localized refactor in graph commit logic; multimodal and routed-expert behavior are preserved by construction, with no auth or data-model changes.

Overview
_commit_turn no longer builds a full (node_id, message) path on every turn. Text-only commits track new nodes in new_node_ids as they are created and skip the O(prefix) list/tuple work that _attribute_mm would not use anyway.

When multi_modal_data is present, the reused prefix is materialized into mm_path and extended for new input messages so image cursor attribution is unchanged. Routed-expert slicing still uses the same new_node_ids ordering (new inputs, then the assistant node).

Reviewed by Cursor Bugbot for commit 8cf96d9. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Skip multimodal path allocation in _commit_turn on text-only turns

In graph.py, _commit_turn previously built a unified path list and called _attribute_mm unconditionally, even when no multimodal data was present. This refactor splits path into new_node_ids (always tracked) and mm_path (only materialized when multi_modal_data exists), and gates the _attribute_mm call on mm_path being non-None.

Macroscope summarized 8cf96d9.

@macroscopeapp

macroscopeapp Bot commented Jun 21, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved

Performance optimization that avoids allocating multimodal path structures when processing text-only turns. The change is self-contained within _commit_turn and preserves existing behavior for multimodal cases while eliminating unnecessary allocations for the common text-only path.

You can customize Macroscope's approvability policy. Learn more.

@xeophon
xeophon merged commit 04131a5 into feat/nano-as-v1 Jun 21, 2026
5 checks passed
pull Bot pushed a commit to Stars1233/verifiers that referenced this pull request Jun 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant