Skip to content

fix(desktop): fitted browser snapshots no longer tile the page - #821

Merged
incognitojam merged 1 commit into
mainfrom
fix/preview-snapshot-tiling
Oct 14, 2026
Merged

incognitojam merged 1 commit into
mainfrom
fix/preview-snapshot-tiling

Conversation

@incognitojam

Copy link
Copy Markdown
Owner

Note

Fitted desktop Browser snapshots now capture the complete displayed page instead of repeating the guest surface. Captures use the displayed surface's resolution; annotation coordinates remain logical page coordinates.

When a freeform viewport is larger than the Browser panel, the fork fits the Electron webview with device-metrics emulation. Page.captureScreenshot then tiles the smaller guest surface into the requested logical-sized bitmap. The page layout and interactive-element coordinates remain correct, but both inline snapshots and saved PNGs repeat the top-left page patch.

Use the existing full-surface webContents.capturePage() path for fitted viewports and map annotation crop rectangles into the captured image's pixels. Snapshots, saved screenshots, annotation crops, and the DevTools fallback share that path. Remove the broken CDP capture helpers and update the fork ledger.

Add an intake note for upstream's server-browser migration: its relayed CDP screenshot would reproduce the fitted desktop capture problem unless intake preserves the full-surface path. Upstream retains a full-size webview under a CSS transform, so its presentation differs from this fork's.

Validation:

  • An isolated Electron 44.4.2 webview harness with a synthetic coordinate grid reproduces 519×359 repeated patches for a 1100×760 logical viewport. Full-surface capture returns the complete, untiled page at 1038×718 on Retina. A logical crop at (800, 400) selects the expected grid cells. Electron 41.5.0 also reproduced the original tiling.
  • All 105 Manager.test.ts tests pass on the final branch. The replacement fitted-capture and crop tests fail against the old implementation and pass with the fix. Existing tests cover adjacent native capture and DevTools fallback behavior.
  • Desktop typecheck and targeted formatting pass. Targeted lint passes with the existing ClipboardItem mock-class warning.
  • A full in-app pass with the Browser panel visible could not be verified: this thread's Browser tools continue to report visible:false. The images below come from the isolated Electron harness, not an in-app capture of the patched desktop client.

The trade-off is capture resolution: fitted captures contain the displayed surface's pixels, which may be fewer than the requested logical viewport dimensions, especially on 1× displays. This affects desktop-owned Browser capture flows; it does not add a contract, provider, entry point, or mobile navigation change.

Before: tiled CDP capture After: full-surface capture
Synthetic grid repeats at the panel-size boundaries Synthetic grid spans the full logical page without repeating

Logical annotation crop at (800, 400):

Crop starts at the expected 800,400 grid cell


Written by an agent (styal Codex harness, gpt-6.1-sol). Implementation and initial validation by an agent (styal Claude harness, claude-opus-5-5).

A fitted <webview> guest only has its on-screen surface pixels. CDP
screenshots of the logical viewport tiled that surface into the output
instead of rendering beyond it. Capture the displayed surface with
capturePage and map logical annotation crops into its pixels, as the
DevTools fallback already did.
@github-actions

Copy link
Copy Markdown

Upstream divergence

Both sides are compared with upstream at the in-order tip 33b7d7b3ae. Excludes .repos/ and pnpm-lock.yaml.

Measure Base This PR Change
Upstream files the fork changes 1,361 1,361 0
Lines changed in them 61,722 61,615 −107
Upstream files the fork removes 41 41 0
Fork-only files 429 429 0
Fork-only lines 74,234 74,234 0
Upstream files whose difference changes (2)
File Change Now differs by
apps/desktop/src/preview/Manager.ts −85 513 lines
apps/desktop/src/preview/Manager.test.ts −22 611 lines

@incognitojam
incognitojam enabled auto-merge (squash) October 14, 2026 17:19
@incognitojam
incognitojam merged commit 6805a40 into main Oct 14, 2026
26 checks passed
@incognitojam
incognitojam deleted the fix/preview-snapshot-tiling branch October 14, 2026 17:23
@github-actions

Copy link
Copy Markdown

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire 4.9 KiB 4.9 KiB +40 B (+0.8%) 6.8 KiB ✅
Codex Thread snapshot wire 3.7 KiB 3.7 KiB 0 B (0.0%) 4.9 KiB ✅
Codex Live turn WebSocket wire 1.1 KiB 1.2 KiB +40 B (+3.4%) 2.0 KiB ✅
Codex Live turn WebSocket decoded 20.4 KiB 20.4 KiB +41 B (+0.2%) 29.3 KiB ✅
Codex Live turn messages 1 2 +1 (+100.0%) 8 ✅
Claude Total thread wire 4.9 KiB 4.9 KiB 0 B (0.0%) 6.8 KiB ✅
Claude Thread snapshot wire 3.7 KiB 3.7 KiB 0 B (0.0%) 4.9 KiB ✅
Claude Live turn WebSocket wire 1.2 KiB 1.2 KiB 0 B (0.0%) 2.0 KiB ✅
Claude Live turn WebSocket decoded 20.8 KiB 20.8 KiB 0 B (0.0%) 29.3 KiB ✅
Claude Live turn messages 2 2 0 (0.0%) 8 ✅

Baseline: 502bb9b · PR result: fea9788 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 106.1 KiB
  • Claude decoded thread snapshot: 106.4 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant