Skip to content

fix(cua-driver-rs)(windows): apply PMv2 DPI manifest (RT_MANIFEST type 24) and remove double scaling in capture paths - #1883

Merged
f-trycua merged 2 commits into
trycua:mainfrom
tobitege:1879-fix-pmv2-dpi-manifest
Jun 16, 2026
Merged

fix(cua-driver-rs)(windows): apply PMv2 DPI manifest (RT_MANIFEST type 24) and remove double scaling in capture paths#1883
f-trycua merged 2 commits into
trycua:mainfrom
tobitege:1879-fix-pmv2-dpi-manifest

Conversation

@tobitege

@tobitege tobitege commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Fixes cua-driver-rs (Windows): PMv2 DPI manifest from #1821 is never applied - RT_MANIFEST embedded as string-typed resource #1879: the Per-Monitor V2 DPI manifest added in fix(cua-driver): add DPI awareness manifest for Windows #1821 was embedded as a custom string-typed resource (RT_MANIFEST is not an .rc keyword), so the Windows loader never applied it and cua-driver.exe kept starting DPI-unaware. The .rc script now uses the numeric resource type 24 (RT_MANIFEST) with ID 1, which the loader actually picks up.
  • Removes the now-incorrect DPI / 96 scaling in the three Windows capture paths (screenshot_via_screen_region, screenshot_window_bytes_with_occlusion_unsafe, screenshot_display_bytes). Under PMv2, GetWindowRect / GetSystemMetrics return physical pixels - the same unit BitBlt / PrintWindow operate in - so scaling them again would shift and oversize captures by the display scale factor once the manifest loads.
  • Corrects the misleading comments in build.rs and the get_screen_size tool that claimed these APIs return logical pixels under PMv2.

All changes are confined to Windows-only code: the platform-windows crate is only compiled on Windows (cfg-gated dependency), and the .rc resource is only embedded inside the #[cfg(target_os = windows)] build step. macOS and Linux are untouched.

Verification (Windows 11, 3840x2160 @ 125%)

  • Built binary contains resource type 24, ID 1; running process reports GetProcessDpiAwareness = 2 (PER_MONITOR). The installed 0.5.2 binary as a negative control: string-typed resource only, awareness 0.
  • get_screen_size now returns 3840x2160 @ 1.25x (was 3072x1728 @ 1x virtualized).
  • Window capture of a UWP/WinUI app (Calculator) positioned away from (0,0) returns the window pixel-exact - no shifted region, no black margins, reported dimensions match the window bounds.
  • cargo check / cargo build clean; unit tests pass. The 6 failing mcp_protocol_test cases fail identically on unpatched main in this environment (pre-existing, unrelated to this change).

Summary by CodeRabbit

  • Bug Fixes
    • Improved screenshot capture alignment and click positioning accuracy across Windows systems with multi-monitor and high-DPI configurations.
    • Fixed coordinate handling to ensure captured screen content and input targets properly align with physical display pixels.
    • Corrected manifest resource definition for proper Windows application initialization.

…4 and drop double scaling in capture paths

The .rc script declared the manifest type as the identifier RT_MANIFEST,
which the resource compiler emits as a custom string-typed resource the
Windows loader never applies - the daemon kept starting DPI-unaware.
Use the numeric type 24 so the Per-Monitor V2 manifest actually loads.

With PMv2 active, GetWindowRect/GetSystemMetrics return physical pixels,
so remove the DPI/96 scaling in screenshot_via_screen_region,
screenshot_window_bytes_with_occlusion_unsafe and screenshot_display_bytes
that would otherwise shift/oversize captures by the scale factor.

Fixes trycua#1879
@vercel

vercel Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

@tobitege is attempting to deploy a commit to the Cua Team on Vercel.

A member of the Team first needs to authorize it.

@coderabbitai

coderabbitai Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: d9b3b0d2-dc80-415d-9346-13dd7b2b20dd

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

This PR fixes Per-Monitor V2 DPI awareness embedding and eliminates double-scaling in Windows screenshot capture. It corrects the manifest resource compiler syntax, removes compensating DPI scaling from three capture paths now that Windows APIs return physical coordinates directly, and updates documentation to reflect physical-pixel semantics.

Changes

Per-Monitor V2 DPI Awareness and Capture Coordinate Alignment

Layer / File(s) Summary
Per-Monitor V2 DPI manifest resource embedding
libs/cua-driver/rust/crates/cua-driver/cua-driver.rc, libs/cua-driver/rust/crates/cua-driver/build.rs
Resource compiler manifest definition is changed from RT_MANIFEST token to numeric type 1 24 so the Per-Monitor V2 manifest actually loads at Windows loader's expected type. Build-file comment is updated to specify Per-Monitor V2 and document resulting physical-pixel behavior.
Capture sizing for Per-Monitor V2 physical pixel coordinates
libs/cua-driver/rust/crates/platform-windows/src/capture.rs
Screen-region BitBlt fallback, window-sized PrintWindow buffer setup, and full-display capture all remove GetDpiForWindow / GetDpiForSystem scaling because Windows APIs now return physical pixels directly under Per-Monitor V2 awareness. Comments clarify the physical-pixel coordinate assumption.
Coordinate space documentation clarification
libs/cua-driver/rust/crates/platform-windows/src/tools/impl_.rs
GetScreenSizeTool documentation is updated to specify that SM_CXSCREEN/SM_CYSCREEN return physical pixels, aligning with screenshot and click-target coordinate space under Per-Monitor V2 DPI awareness.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related issues

  • trycua/cua#1879: Addresses the root cause — Per-Monitor V2 manifest resource was declared with string-typed RT_MANIFEST instead of numeric type 24, and capture paths were double-scaling under the (previously-unloaded) PMv2 regime.

Possibly related PRs

  • trycua/cua#1821: Added the Per-Monitor V2 DPI-awareness manifest embedding; this PR ensures it loads correctly and fixes the double-scaling in capture paths that was premised on PMv2 coordinate behavior.
  • trycua/cua#1697: Modified the same PrintWindow buffer sizing and GetWindowRect-based capture logic in platform-windows/src/capture.rs that this PR updates for physical pixel coordinates.
  • trycua/cua#1599: Modified the screen-region BitBlt fallback in screenshot_via_screen_region that this PR updates to remove DPI scaling.

Poem

A manifest hops through compile to land,
No longer lost in string-typed wonderland—
Pixels stay physical when PMv2 calls,
No double-scaling ghosts within these walls.
Per-monitor wisdom, at last, takes flight! 🐰✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main changes: fixing PMv2 DPI manifest embedding (RT_MANIFEST type 24) and removing double scaling in capture paths.
Linked Issues check ✅ Passed The PR addresses all key coding objectives from #1879: fixing manifest embedding as type 24 ID 1, removing DPI scaling from three capture paths, and updating misleading comments.
Out of Scope Changes check ✅ Passed All changes are directly scoped to issue #1879: manifest embedding fix, capture path scaling removals, and documentation updates. No unrelated changes detected.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
libs/cua-driver/rust/crates/platform-windows/src/tools/impl_.rs (1)

3886-3894: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Fix coordinate-space wording mismatch in get_screen_size output.

The updated comment correctly says these are physical pixels, but the returned message still says points (Line 3894). That makes the tool output contradictory and can mislead callers about click/screenshot coordinate space.

Suggested patch
-        ToolResult::text(format!("✅ Main display: {w}x{h} points @ {scale}x"))
+        ToolResult::text(format!("✅ Main display: {w}x{h} physical pixels @ {scale}x"))
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@libs/cua-driver/rust/crates/platform-windows/src/tools/impl_.rs` around lines
3886 - 3894, The tool prints a misleading unit: the code computes physical pixel
dimensions (GetSystemMetrics) but the message says "points"; update the output
in the block around GetSystemMetrics/GetDpiForSystem (the get_screen_size logic
that returns ToolResult::text(...)) to use "pixels" instead of "points" (e.g.,
change the formatted string emitted by ToolResult::text to "Main display:
{w}x{h} pixels @ {scale}x") so the wording matches the comment and actual
coordinate space.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@libs/cua-driver/rust/crates/platform-windows/src/tools/impl_.rs`:
- Around line 3886-3894: The tool prints a misleading unit: the code computes
physical pixel dimensions (GetSystemMetrics) but the message says "points";
update the output in the block around GetSystemMetrics/GetDpiForSystem (the
get_screen_size logic that returns ToolResult::text(...)) to use "pixels"
instead of "points" (e.g., change the formatted string emitted by
ToolResult::text to "Main display: {w}x{h} pixels @ {scale}x") so the wording
matches the comment and actual coordinate space.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: f5889ffb-fec2-4eb0-a012-3a0b3397b4f9

📥 Commits

Reviewing files that changed from the base of the PR and between 2925b49 and ee7ac50.

📒 Files selected for processing (4)
  • libs/cua-driver/rust/crates/cua-driver/build.rs
  • libs/cua-driver/rust/crates/cua-driver/cua-driver.rc
  • libs/cua-driver/rust/crates/platform-windows/src/capture.rs
  • libs/cua-driver/rust/crates/platform-windows/src/tools/impl_.rs

… not points

The values are physical pixels under PMv2; the tool text and description
still said points. Update the wording and the parity example parser.
@f-trycua
f-trycua merged commit fbc4a21 into trycua:main Jun 16, 2026
1 of 2 checks passed
@f-trycua

Copy link
Copy Markdown
Collaborator

Merged — thanks @tobitege, excellent investigation. Root cause was spot on: RT_MANIFEST isn't an .rc keyword so it was emitted as a string-typed resource the loader never applied, leaving the process DPI-unaware — which is exactly why the capture paths had compensating DPI/96 scaling. Coupling the manifest fix (numeric type 24) with removing that scaling in the same commit is the right call. Agent-reviewed before merge: confirmed those three are the only capture coordinate-scaling sites, GetScreenSizeTool correctly keeps GetDpiForSystem only for the reported scale_factor, and no click path scales by DPI (clicks use the separate resize ratio), so nothing double-scales now. Appreciate the 4K@125% hardware validation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cua-driver-rs (Windows): PMv2 DPI manifest from #1821 is never applied - RT_MANIFEST embedded as string-typed resource

2 participants