feat(cua-driver-rs): screenshot defaults — jpeg @ 85 + max_image_dimension 1568 by f-trycua · Pull Request #1663 · trycua/cua

f-trycua · 2026-05-23T12:23:17Z

Summary

Two behavior changes to the `screenshot` tool on all 3 Rust platforms, motivated by the Windows VM dogfood overnight finding that screenshots were coming back at ~10.5 MB:

Setting	Before	After	Why
`max_image_dimension` default	`0` (no cap)	`1568`	Swift parity (`CuaDriverConfig.defaultMaxImageDimension` is 1568). The 0 default was a Rust-only regression.
`screenshot` tool `format` default	`png`	`jpeg`	Agents want compact images for vision-context. Schema still accepts both; PNG is opt-in via `{"format":"png"}`. Diverges from Swift (Swift still `png`).
`screenshot` tool `quality` default	`95` (macos+windows) / `85` (linux)	`85` everywhere	Sweet spot for screen content; q=95 was actually larger than PNG for solid-color UI. Linux already at 85; macOS Claude-Code-compat tool also 85.

Measured impact (macOS, 3840×2160 → 1568×882)

Config	Decoded bytes
before tonight (PNG, no cap)	4,866,498
jpeg @ 85, no cap	825,429 (5.9×)
jpeg @ 85, cap 1568	167,895 (29×)

Same query on Windows VM: ~10.5 MB → 85 KB (125× reduction).

Files

`crates/platform-{windows,linux,macos}/src/tools/{impl_,screenshot}.rs` — defaults + tool descriptions
`crates/platform-{windows,linux,macos}/src/tools/{impl_,mod}.rs` — `DriverConfig::default().max_image_dimension`
`tests/integration/test_cursor_visibility.py` — pass explicit `format=png` so the per-pixel cursor-colour decode keeps working
`crates/cua-driver/tests/mcp_protocol_test.rs` — outdated docstring
`PARITY.md` — moves "quality 95" out of Fixed and adds two Intentional Rust-only rows; new Fixed row for max_image_dimension

Compat

The CLI's `screenshot_png_b64` structuredContent key keeps its name — it's a cross-implementation contract (Swift `AppState.swift:97`, get_window_state, both SKILL.md files, all integration tests). The companion `screenshot_mime_type` field reports the actual MIME (`image/jpeg` for the new default).

Users with a persisted `~/Library/Application Support/cua-driver/config.json` that explicitly sets `max_image_dimension` (even to 0 for "no cap") keep their value — the config loader only falls back to the default when the field is absent. Fresh installs get 1568.

Docs

`mcp-tools.mdx` is auto-generated from Swift sources, which still default to PNG — not regenerated in this PR. The docs will become accurate when Swift adopts the same defaults (tracked as a follow-up parity question).
`PARITY.md` updated this PR.

Test plan

`python3 -m unittest test_cli` → 17/17 passing on macOS
`cargo test cursor-overlay::util::tests` 4/4 + `cua-driver stdin_bom` 2/2 on Windows VM
Live verification on Windows VM (cuademo Session 9): screenshot via daemon went from 10.5 MB → 85 KB
CI green
Reviewer sanity-check: do we want to also update Swift to JPEG default in a follow-up PR, or leave Swift as PNG?

🤖 Generated with Claude Code

Summary by CodeRabbit

Release Notes

Bug Fixes
- Updated screenshot defaults: format changed from PNG to JPEG with quality 85 across Windows, macOS, and Linux.
- Set maximum image dimension default to 1568 pixels.
Tests
- Updated integration tests to align with new screenshot format defaults.

All 3 platforms now return JPEG by default instead of PNG when callers omit `format`. Quality also drops from 95 → 85 (already Linux's default and the macOS Claude-Code-compat tool's default). For UI screenshots (lots of solid colors + text) the size impact is small or even neutral — PNG's deflate compresses flat regions very well. The big wins materialize for photographic content (web pages, docs with images) and at lower quality settings agents typically request. The CLI's `screenshot_png_b64` key name stays as-is — it's a cross-implementation contract with Swift (AppState.swift:97, GetWindowStateTool.swift, both SKILL.md files, all integration tests). The `screenshot_mime_type` companion key reports the actual MIME. test_cursor_visibility.py now passes `format=png` explicitly so the per-pixel cursor-colour decode keeps working. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

DriverConfig::default().max_image_dimension was 0 (no cap) on all 3 Rust platforms, while Swift's CuaDriverConfig.defaultMaxImageDimension is 1568. That divergence is why Windows-VM screenshots came back at ~10MB — full-resolution PNGs of a 1512x949 desktop. After this change, screenshots are downscaled (long edge) to 1568px before encoding, matching Swift. Combined with the prior jpeg-default commit, the size impact on a 3840x2160 macOS screenshot is: before tonight: 4,866,498 bytes (full-res PNG) jpeg default: 825,429 bytes (5.9×) + cap 1568: 167,895 bytes (29×) Users with a persisted config.json that explicitly sets `max_image_dimension` (even to 0 for "no cap") keep their value — the config loader at tools/mod.rs:150 only falls back to the default when the field is absent. Fresh installs get the new 1568 default. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Updates the `screenshot` tool entry to: - Replace the "Default JPEG quality 95 (matches Swift)" Fixed item (the prior parity work) with new wording acknowledging the move back to 85 as intentional divergence. - Add a Fixed item for `max_image_dimension` default 0 → 1568, which IS now Swift-parity. - Add two new "Intentional Rust-only" items: default `format=jpeg` (Swift still `png`) and default `quality=85` (Swift `95`). No code change in this commit; the source changes are in the two prior commits on this branch. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

vercel · 2026-05-23T12:23:21Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
docs	Ignored	Preview	May 23, 2026 12:23pm

coderabbitai · 2026-05-23T12:23:35Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 61ea57eb-b6f7-446c-a574-e889410068a2

📥 Commits

Reviewing files that changed from the base of the PR and between e37687c and 63bfb73.

📒 Files selected for processing (7)

libs/cua-driver-rs/PARITY.md
libs/cua-driver-rs/crates/cua-driver/tests/mcp_protocol_test.rs
libs/cua-driver-rs/crates/platform-linux/src/tools/impl_.rs
libs/cua-driver-rs/crates/platform-macos/src/tools/mod.rs
libs/cua-driver-rs/crates/platform-macos/src/tools/screenshot.rs
libs/cua-driver-rs/crates/platform-windows/src/tools/impl_.rs
libs/cua-driver-rs/tests/integration/test_cursor_visibility.py

📝 Walkthrough

Walkthrough

This PR unifies screenshot tool defaults across all platforms (Linux, macOS, Windows) by setting JPEG as the default format with quality 85, capping image dimensions to 1568 pixels, and updating documentation and tests accordingly.

Changes

Screenshot Default Format and Dimension Updates

Layer / File(s)	Summary
Image dimension defaults across platforms `libs/cua-driver-rs/crates/platform-linux/src/tools/impl_.rs`, `libs/cua-driver-rs/crates/platform-macos/src/tools/mod.rs`, `libs/cua-driver-rs/crates/platform-windows/src/tools/impl_.rs`	`DriverConfig::default()` now sets `max_image_dimension` to `1568` instead of `0` across all platform implementations, with updated field documentation explaining the dimension cap applied during screenshot/zoom operations.
Screenshot tool format and quality defaults `libs/cua-driver-rs/crates/platform-linux/src/tools/impl_.rs`, `libs/cua-driver-rs/crates/platform-macos/src/tools/screenshot.rs`, `libs/cua-driver-rs/crates/platform-windows/src/tools/impl_.rs`	Tool descriptions, input schemas, and invocation defaults updated across all platforms: `format` now defaults to `"jpeg"` (from `"png"`), `quality` now defaults to `85` (from `95`), and schema documentation clarifies that `quality` is JPEG-specific and ignored for PNG.
Documentation and test updates `libs/cua-driver-rs/PARITY.md`, `libs/cua-driver-rs/crates/cua-driver/tests/mcp_protocol_test.rs`, `libs/cua-driver-rs/tests/integration/test_cursor_visibility.py`	PARITY.md updated to document Rust-specific format/quality defaults as intentional divergences from Swift; test comment clarified to reflect image content return (not PNG-specific); integration test modified to explicitly request PNG for lossless pixel-level color detection.

🎯 2 (Simple) | ⏱️ ~10 minutes

A screenshotful change hops through the wood,
From PNG to JPEG, dimensioned good,
Cross-platform defaults now unified true,
Documentation and tests polished anew! 🐰📸

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 27.27% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main changes: JPEG format default @ 85 quality and max_image_dimension default of 1568, matching the PR's primary objectives.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch feat/cua-driver-rs-screenshot-defaults-jpeg-and-size-cap

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

f-trycua and others added 3 commits May 23, 2026 14:22

f-trycua merged commit dd6b521 into main May 23, 2026
5 checks passed

f-trycua deleted the feat/cua-driver-rs-screenshot-defaults-jpeg-and-size-cap branch May 23, 2026 14:39

coderabbitai Bot mentioned this pull request Jun 26, 2026

fix(cua-driver/windows): persist config values for config parity #2033

Closed

Navigation Menu

Uh oh!

Uh oh!

feat(cua-driver-rs): screenshot defaults — jpeg @ 85 + max_image_dimension 1568 - #1663

feat(cua-driver-rs): screenshot defaults — jpeg @ 85 + max_image_dimension 1568#1663
f-trycua merged 3 commits into
mainfrom
feat/cua-driver-rs-screenshot-defaults-jpeg-and-size-cap

f-trycua commented May 23, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

vercel Bot commented May 23, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented May 23, 2026 •

edited

Loading

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

f-trycua commented May 23, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Measured impact (macOS, 3840×2160 → 1568×882)

Files

Compat

Docs

Test plan

Summary by CodeRabbit

Release Notes

Uh oh!

vercel Bot commented May 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot commented May 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

f-trycua commented May 23, 2026 •

edited by coderabbitai Bot

Loading

vercel Bot commented May 23, 2026 •

edited

Loading

coderabbitai Bot commented May 23, 2026 •

edited

Loading