fix(computer-use): cap elements array in aux-vision routing path

[AhmetArif0]

Root Cause

_route_capture_through_aux_vision returned cap.elements verbatim —
bypassing the max_elements cap that _capture_response computes for the
AX and multimodal paths.

On a dense UI (Electron app, Slack, JetBrains IDE), SOM captures produce
600+ AX nodes. When auxiliary.vision is configured or the main model is
not vision-capable, those captures route through _route_capture_through_aux_vision,
which was returning all nodes uncapped. The AX-only path was fixed by #30145,
but the aux-vision routing path was not updated symmetrically.

Failure Path

1. User has auxiliary.vision configured (or main model doesn't support vision).
2. Agent calls computer_use(action="capture", mode="som") on a dense UI (600 elements).
3. _capture_response computes visible_elements = cap.elements[:max_elements] (100 nodes).
4. _should_route_through_aux_vision() returns True.
5. _route_capture_through_aux_vision(cap, summary) is called with the full cap object.
6. Return statement: "elements": [_element_to_dict(e) for e in cap.elements] — all 600 nodes emitted, context blowup.

Fix

Pass visible_elements (already capped) from _capture_response to
_route_capture_through_aux_vision. Add total_elements and conditional
truncated_elements fields for parity with the AX path so the model knows
the response is partial.

-routed = _route_capture_through_aux_vision(cap, summary)
+routed = _route_capture_through_aux_vision(cap, summary, visible_elements)

-"elements": [_element_to_dict(e) for e in cap.elements],
+"elements": [_element_to_dict(e) for e in _effective_elements],
+"total_elements": _total,
 "summary": summary,
+# "truncated_elements": _truncated  (only when truncated)

Symmetric with the AX path fix in #30145. No behaviour change for captures
with fewer than max_elements nodes.

Tests

3 regression tests added to test_computer_use_capture_routing.py:

• 600-element SOM capture routed via aux vision is capped to _DEFAULT_MAX_ELEMENTS (100)
• Explicit max_elements=50 override flows through to the aux-vision response
• Captures under the cap do not surface a truncated_elements field

All 90 computer-use tests pass (test_computer_use.py + test_computer_use_capture_routing.py).

[teknium1]

Thanks for tracing the cap bypass; current main still has it: tools/computer_use/tool.py:539-541 computes visible_elements, while the auxiliary route is called without it at tools/computer_use/tool.py:587 and serializes cap.elements at tools/computer_use/tool.py:829.

Problems

• The new dense fixtures use a 1×1 PNG. Current main treats images below 8×8 as image_too_small (tools/computer_use/tool.py:542-550) and skips the auxiliary route (tools/computer_use/tool.py:579). The proposed tests would therefore pass through the existing text fallback rather than exercise this change.

Suggested changes

• Use the 8×8 fixture already defined at tests/tools/test_computer_use_capture_routing.py:36-40, then assert vision_analysis_routed_via == "auxiliary.vision" or verify the vision mock was called.
• Restore the existing image_url assertion to TestBugReproductionAnchor; the diff currently leaves it attached to the new under-cap test.

Automated hermes-sweeper review.

[GottZ]

This was generated by AI during triage.

Summary

Three PRs touch computer-use vision routing, but they address distinct causes: #30426 caps elements after auxiliary-vision routing, #42158 revises CUA window discovery and capture transport while including auxiliary-routing fixes already on main, and #51551 enables native vision transport for OpenCode providers.

Related pull requests

• #30426 related — (+124/-4) — keep open, fix tests, then merge: the production diff correctly passes the already capped visible_elements into the auxiliary route and reports truncation metadata, directly fixing the uncapped dense-UI response. The keep_open review identifies blocking test defects: the 1×1 fixture bypasses auxiliary routing, and an existing image_url assertion was accidentally moved.
• #42158 related — (+163/-36) — keep open for rebase and scope reduction: its cache-directory and auxiliary-analysis handling overlap functionality already present on main, while its remaining CUA changes include off-screen windows and unconditionally use get_window_state. As the keep_open contributor review notes, that conflicts with current on-screen selection semantics and bypasses the legacy-screenshot capability path, so it needs a targeted reproduction and mixed-inventory coverage before consideration.
• #51551 related — (+78/-2) — keep open as an independent merge candidate: the diff marks opencode-go and opencode-zen as vision-capable at the shared transport gate, preventing vision-capable models from being incorrectly routed through auxiliary text analysis. This directly addresses the live provider-profile cause identified by the keep_open review and includes scoped tests for both vision_analyze and computer-use routing.

Suggested consolidation

Merge #30426 after replacing the undersized image fixture, proving auxiliary.vision was actually invoked, and restoring the displaced assertion. Evaluate #51551 independently because it fixes a separate provider-capability gate; rebase and narrow #42158 around a proven off-screen-window defect while preserving current capability routing. None of #30426, #42158, or #51551 should be closed as duplicates of another.

Cross-PR triage: Reviewed 3 pull requests and 0 issues in this complex. Each diff was read against this issue; Assessment working set: 27 kB of PR diffs, 12 kB of issue/PR text, 4 kB of discussion (4 comments), 0 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.