Skip to content

fix(computer-use): pass capture_mode to get_window_state for vision/som captures - #1

Open
kumaxs wants to merge 1 commit into
mainfrom
fix/computer-use-capture-mode
Open

fix(computer-use): pass capture_mode to get_window_state for vision/som captures#1
kumaxs wants to merge 1 commit into
mainfrom
fix/computer-use-capture-mode

Conversation

@kumaxs

@kumaxs kumaxs commented Jul 10, 2026

Copy link
Copy Markdown
Owner

Summary

cua-driver 0.5.x+ requires capture_mode in get_window_state calls to return a screenshot image. Without it, the driver only returns the accessibility tree — no PNG — causing computer_use captures (both vision and SOM modes) to silently return empty 0×0 results.

PR NousResearch#50994 and NousResearch#51078 fixed the routing (using get_window_state instead of the removed screenshot tool), but neither added the required capture_mode parameter. The fallback paths continued to produce empty captures.

Changes

Add capture_mode to all 4 get_window_state call sites in tools/computer_use/cua_backend.py:

Location capture_mode value
Vision MCP fallback (line ~1412) "vision"
Vision CLI re-fetch (line ~1440) "vision"
SOM/ax MCP path (line ~1458) mode (dynamic)
SOM/ax CLI re-fetch (line ~1495) mode (dynamic)

Related

Verification

  • Syntax validated
  • All 4 calls now consistently pass capture_mode
  • SOM path uses mode (variable) so both "som" and "ax" modes work correctly
  • No API change, no new dependencies

hermes-agent-contributors
fix(computer-use): pass capture_mode to get_window_state for vision/s…
567d71e 
…om captures

cua-driver 0.5.x+ requires capture_mode in get_window_state calls to
return a screenshot image. Without it, the driver returns only the AX
tree, causing computer_use capture (both vision and SOM modes) to
silently return an empty 0x0 result.

Add capture_mode to all 4 get_window_state call sites:
- vision MCP fallback path → capture_mode='vision'
- vision CLI re-fetch path → capture_mode='vision'
- som/ax MCP path → capture_mode=mode
- som/ax CLI re-fetch path → capture_mode=mode

Fixes NousResearch#39242 (partial — the screenshot tool routing was already fixed
by PR NousResearch#50994 / NousResearch#51078, but the missing capture_mode parameter
continued to produce empty captures on the fallback paths).
@kumaxs
kumaxs force-pushed the fix/computer-use-capture-mode branch from 4448ac5 to 567d71e Compare July 12, 2026 04:08
kumaxs pushed a commit that referenced this pull request Aug 4, 2026
@teknium1
test(gateway): real-DB restart regression for persisted hygiene coold…
021d191 
…own (NousResearch#74136)

Fix-up for the cherry-picked cooldown persistence: the PR's tests mocked
the DB (SimpleNamespace(_db=MagicMock())), which cannot prove the cooldown
survives a restart. Replace with the production shape — a real SessionDB
on disk behind the real AsyncSessionDB facade — and add a restart
regression: fail a hygiene compression on runner #1, tear it down, build a
fresh GatewayRunner on the SAME database, and assert the cooldown is still
honored (no compression agent instantiated). Also updates the timeout test
to assert the DB-backed record_compression_failure_cooldown write instead
of the removed in-memory dict.

Sabotage-verified: reverting gateway/run.py to the in-memory dict makes
the restart test fail.
kumaxs pushed a commit that referenced this pull request Aug 4, 2026
@teknium1
docs(site): redirect /quickstart and /installation short paths
9467e99 
Users following abbreviated links guess /docs/quickstart and
/docs/installation and hit raw GitHub-Pages 404s — the real pages live
under /docs/getting-started/. Add client redirects for both.

Consumer-onboarding audit finding #1, Aug 2026.
kumaxs pushed a commit that referenced this pull request Aug 4, 2026
@teknium1
feat(patch): detect already-applied edits and return success no-op
99d6f55 
The #1 patch failure class in production (state.db mining, 250k-window)
is a re-send of an edit that already landed: 'old_string and new_string
are identical' (299 occurrences) plus a share of hunk-not-found errors
where the new text is already in the file. These errored, sending
models into re-read/re-patch loops.

New tools/fuzzy_match.is_already_applied(content, old, new) — a
conservative check requiring (1) non-trivial new_string (>=8 chars),
(2) EXACT presence of new_string, (3) old_string gone (unless
identical). Wired into three sites:

- patch_replace (replace mode): returns success + no_change: true +
  an explicit note instead of the identical-strings / no-match error.
- V4A validation phase: an already-applied hunk validates as a no-op
  so multi-hunk patches no longer fail wholesale when one hunk landed
  in a prior call.
- V4A apply phase: mirrors the same skip so the two phases agree.

Genuine no-matches (new text absent) and half-applied renames (old
text still present) keep their error behavior — covered by tests.
kumaxs pushed a commit that referenced this pull request Aug 4, 2026
@teknium1
fix(process): wait timeout result reads as status, not failure
0b149ca 
process(action='wait') hitting its window returned status='timeout'
with a terse note — models read it as an error and re-issued identical
waits (process is the #1 exact-duplicate tool call in production: 511
dupes in a 400k-msg window; wait is 57% of all process actions).

The timeout result now carries:
- process_running: true — machine-readable 'this is a status, not a
  failure'
- an explicit note: 'Wait window of Ns elapsed — the process is still
  running. This is not an error. Uptime: Ms.' plus the right next step:
  when notify_on_complete is set, 'you will be notified on exit — do
  more work instead of waiting again'; otherwise a pointer to
  notify_on_complete for next time.
- the clamp note (requested > max) now composes with the status note
  instead of replacing it.

Exited/interrupted results are unchanged.
kumaxs pushed a commit that referenced this pull request Aug 4, 2026
@teknium1
fix(agent): never release the durable compression lease mid-commit (r…
0277cc4 
…e-review #1)

revoke_commit_admission() used to invoke the holder-qualified lease
release unconditionally — including while an admitted commit was still
mutating SessionDB — letting a second compressor acquire the durable
lock mid-commit and interleave with the first commit's writes.

The admission_revoked flag store stays lock-free, but the lease-release
decision now coordinates with the fence lock:
- revoke acquires the fence lock non-blocking; on success no commit can
  be in flight (an admitted commit retains the lock until finish_commit)
  and the release runs immediately, still under the lock so a racing
  begin_commit cannot slip between the check and the release.
- on failure the release is deferred: finish_commit() re-checks
  _admission_revoked and performs it AFTER the commit completes (prompt
  even if the worker thread is later parked), and the begin_commit
  refusal path does the same for a revoke that lost the race to a
  transient lock-setup/cancel boundary. All paths are idempotent with
  the worker's own outer cleanup (DB release is holder-qualified).

Invariant encoded + tested: no second compressor can acquire the durable
lock while an admitted commit is still mutating; after a post-revoke
commit finishes the lease is released promptly. Both regressions
(revoke-during-commit deferral, revoke-before-commit immediate release +
refused begin_commit) are sabotage-verified.
kumaxs pushed a commit that referenced this pull request Aug 25, 2026
@teknium1
test: shim-progress fake hermes answers the --keep-stash --help probe
27d356c 
posix.sh now probes `update --help` before the real update call; the fake
counted the probe as call #1, shifting the exits.N mapping so the retry
gate never fired. Answer the probe out-of-band so counted calls remain
actual update attempts.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@kumaxs