Fix browser panel layout with console open#5
Merged
Conversation
zvadaadam
added a commit
that referenced
this pull request
Feb 23, 2026
1. **inject-mode.ts: var() scan now covers padding/gap (#3)** Renamed colorProps → varScanProps, added padding/gap so getMatchedVarDeclarations captures var() tokens for spacing props. 2. **inject-mode.ts: font-weight fallback added (#10)** Added font-weight to the fallback conditional alongside font-size and border-radius. Filters out 'normal' (CSS default) as noise. 3. **BrowserPanel.tsx: DevTools state races with async invoke (#4)** Moved handleUpdateTab into .then() so devtoolsOpen only updates after the Rust command succeeds. Prevents UI desync on failure. 4. **BrowserTab.tsx: safeListen missing .catch() (#5)** Added .catch() to the listen() promise chain to prevent unhandled rejections if Tauri listener registration fails. 5. **BrowserTab.tsx: injectionFailed not cleared on success (#7)** Changed onUpdateTab to also set injectionFailed: false after successful injection, clearing stale error indicators. 6. **webview.rs: eprintln! gated for release builds (#8)** All 4 eprintln! calls in eval_browser_webview_with_result wrapped with cfg!(debug_assertions). Prevents leaking JS content, URLs, tokens, or page console data in production. 7. **webview.rs: DevTools error propagation (#9)** open/close_browser_devtools now use Arc<Mutex<Option<String>>> to propagate null-pointer errors out of with_webview closures. Frontend receives Err() instead of silent Ok(()) when DevTools unavailable. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
zvadaadam
added a commit
that referenced
this pull request
Feb 23, 2026
* feat: Fix browser inspect element pills with code quality improvements ## Changes ### 1. Fix DevTools Expansion Bug (Tauri Native Layer) When clicking the DevTools button, the browser view would unexpectedly expand. Root cause: `_inspector.show()` docks by splitting WKWebView's superview, resizing the webview. After `detach()` moves inspector to floating window, frame isn't restored. **Fix:** Save WKWebView frame before show(), restore after detach(). - Modified: src-tauri/src/commands/webview.rs (lines 570-582) ### 2. Fix Cursor Animation Attribute Mismatch (Issue #1 - HIGH) Cursor visual effects (move, ripple, pin) were silently failing because they searched for `data-cursor-ref` but elements were marked with `data-hive-ref`. **Fix:** Replace all `data-cursor-ref` references with `data-hive-ref` to match the actual attribute used in inject-mode.ts. - Modified: src/features/browser/automation/visual-effects.ts - Line 18: JSDoc comment - Line 25: buildMoveCursorAndRippleJs() - Line 45: buildPinCursorJs() - Line 96: buildHighlightElementJs() ### 3. Add IME Composition Guard to MessageInput (Issue #2 - MEDIUM) CJK (Chinese, Japanese, Korean) users using Input Method Editors (IME) would lose their input when pressing Enter during composition, since the handler didn't check if composition was in progress. **Fix:** Add `!e.nativeEvent.isComposing` guard to Enter key handler to prevent message send during active IME composition. - Modified: src/features/session/ui/MessageInput.tsx (line 286) ### 4. Fix Element Selector Injection Exception Handling (Issue #3 - MEDIUM) When `verify_injection` script threw an exception, the catch block didn't return, causing exception object to fall through to `eval_browser_webview_with_result`, which would parse it as a result string instead of error. **Fix:** Add explicit `return;` in catch block to prevent fall-through. - Modified: src/features/browser/ui/BrowserTab.tsx (line 521) ### 5. Fix Selector Toggle State on Eval Failure (Issue #4 - MEDIUM) When element selector injection failed, `selectorActive` was set in the UI store but outside the try block, so errors during injection were silently swallowed. **Fix:** Move state update inside try block to fail-fast on injection errors. - Modified: src/features/browser/ui/BrowserTab.tsx (line 563) ### 6. Add Polling Guard to Inspect Event Drain (Issue #5 - MEDIUM) Inspect event drain was polling unconditionally every 200ms, consuming ~5 IPC calls/second unnecessarily. With multiple tabs this compounds quickly. **Fix:** Only poll when selector is active. Add `!tab.selectorActive` guard and add to dependency array. - Modified: src/features/browser/ui/BrowserTab.tsx (lines 396-458) ### 7. Replace if/else with ts-pattern (Issue #6 - LOW) Long if/else chain for event type dispatch violates CLAUDE.md requirement to use ts-pattern for discriminated union dispatch. **Fix:** Import ts-pattern and use `.with()` / `.otherwise()` for event type matching. - Modified: src/features/browser/ui/BrowserTab.tsx (lines 26, 427-444) ### 8. Fix Hover Animation Timing (Issue #7 - LOW) InspectedElementCard hover transition was `duration-150` instead of `duration-200`, violating CLAUDE.md animation standard of 200-300ms default duration. **Fix:** Update to `duration-200 ease` to match project animation guidelines. - Modified: src/features/session/ui/InspectedElementCard.tsx (line 48) ## Verification - TypeScript compilation: ✓ (pre-existing Lucide icon issue unrelated to changes) - All fixes verified and applied by parallel code-reviewer agents - Changes preserve existing functionality while fixing correctness issues - Performance improvement: Eliminates ~5 IPC calls/second from unconditional polling ## Impact - **Bug fixes:** 3 (DevTools expansion, cursor animations, IME composition) - **Correctness improvements:** 2 (exception handling, state sync) - **Performance:** Reduced IPC quota consumption from inspect event drain - **Code quality:** Full CLAUDE.md compliance (ts-pattern, animation timing) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: Address PR review — ts-pattern console dispatch + comment consistency 1. Replace nested ternary on log.l with ts-pattern match() in console drain loop (BrowserTab.tsx:365). Consistent with inspect event drain at line 428 which already uses match(). CLAUDE.md requires ts-pattern for all discriminator field dispatch. 2. Fix contradictory architecture comments across 3 files: - inject/inspect-mode.ts: Remove "DUAL mechanism: PRIMARY title-channel" language and phantom serialization queue details. Buffer+drain via eval_browser_webview_with_result is the sole path. - BrowserTab.tsx: Change "FALLBACK path" to "sole delivery path". - webview.rs: Remove reference to non-existent sendViaTitle(), mark title-channel handler as backward-compat only. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: Address PR review — 7 issues across 4 files 1. **inject-mode.ts: var() scan now covers padding/gap (#3)** Renamed colorProps → varScanProps, added padding/gap so getMatchedVarDeclarations captures var() tokens for spacing props. 2. **inject-mode.ts: font-weight fallback added (#10)** Added font-weight to the fallback conditional alongside font-size and border-radius. Filters out 'normal' (CSS default) as noise. 3. **BrowserPanel.tsx: DevTools state races with async invoke (#4)** Moved handleUpdateTab into .then() so devtoolsOpen only updates after the Rust command succeeds. Prevents UI desync on failure. 4. **BrowserTab.tsx: safeListen missing .catch() (#5)** Added .catch() to the listen() promise chain to prevent unhandled rejections if Tauri listener registration fails. 5. **BrowserTab.tsx: injectionFailed not cleared on success (#7)** Changed onUpdateTab to also set injectionFailed: false after successful injection, clearing stale error indicators. 6. **webview.rs: eprintln! gated for release builds (#8)** All 4 eprintln! calls in eval_browser_webview_with_result wrapped with cfg!(debug_assertions). Prevents leaking JS content, URLs, tokens, or page console data in production. 7. **webview.rs: DevTools error propagation (#9)** open/close_browser_devtools now use Arc<Mutex<Option<String>>> to propagate null-pointer errors out of with_webview closures. Frontend receives Err() instead of silent Ok(()) when DevTools unavailable. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(ci): build inject scripts before sidecar unit tests browser-templates.test.ts imports browser-utils.ts which reads dist-inject/browser-utils.js via ?raw import. The dist-inject/ directory is gitignored and must be built first. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(test): update browser-templates tests for inject refactor - Remove dead BROWSER_UTILS import (renamed to BROWSER_UTILS_SETUP) - Remove incorrect buildPressKeyJs BROWSER_UTILS test (it intentionally doesn't use browser utils — operates on document.activeElement directly) - Update assertIsIIFE to accept both classic IIFEs (function(){...})() and arrow IIFEs (() => {...})() — esbuild format: "iife" emits the latter for compiled inject scripts like VISUAL_EFFECTS_SETUP Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address code review round 3 — security, a11y, concurrency - Gate getReactProps() and getShallowInnerHTML() to local context only; external sites can leak PII/tokens via React fiber props and innerHTML - Remove 'value' from ATTR_WHITELIST (can contain passwords) - Add in-flight guards to console and inspect drain intervals to prevent overlapping async invokes when eval_browser_webview is slow - Add aria-hidden + sr-only text for injection failure indicator - Add aria-label to screenshot button - Remove duplicate comment blocks in inject-mode.ts - Fix stale "title-channel" comment in BrowserTab.tsx Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(browser): add screenshot button + mobile viewport toggle Add two new browser panel toolbar features: 1. Screenshot button (Camera icon): captures WKWebView as JPEG via Rust IPC and attaches to chat input via CustomEvent bridge. 2. Mobile/Desktop viewport toggle (Smartphone/Monitor icon): constrains browser area to 390px (iPhone 14 logical width) centered with mx-auto. The mobile view required replacing the tab stacking strategy — absolute positioning (inset-0) didn't reliably inherit width constraints during parent restructuring, causing getBoundingClientRect() to return stale full-width values. Tabs now stack via CSS Grid ([grid-area:1/1]) which keeps them in normal document flow. A useLayoutEffect with hide → setBounds → show cycle ensures the native WKWebView repositions synchronously on toggle. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
zvadaadam
added a commit
that referenced
this pull request
Mar 2, 2026
…uncation Addresses 3 actionable findings from CodeRabbit review: 1. **Redact raw prompts from logs** (comments #2 + #5): Replace `prompt.slice(0, 80)` with `promptLength` in timing logs in both claude-handler.ts and index.ts. User prompts were being written to /tmp log files — now only the length is logged. 2. **Fix messageCount ReferenceError** (comment #3): Move `let messageCount` declaration from inside the `try` block to before it. In JS, `let` inside `try {}` is NOT accessible in `catch {}` — they are separate block scopes. This caused a ReferenceError in the catch block, masking the actual error. 3. **Safe UTF-8 truncation in Rust** (comment #6): Replace `&error[..error.len().min(80)]` with `error.chars().take(80).collect::<String>()` in socket.rs. Byte-index slicing can panic on multibyte UTF-8 boundaries. (session_id truncation left as-is — UUIDv7 is guaranteed ASCII.) Dismissed as not actionable: - Comment #4 (error redaction in session-writer): Error messages are already sanitized by classifyError(). Redacting would lose debuggability for zero security gain. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
zvadaadam
added a commit
that referenced
this pull request
Mar 2, 2026
…ocess (#163) * fix(session): implement proper cancel/stop flow with force-kill subprocess ## Problem When user clicked stop, two things failed: 1. Agent kept running — `query.interrupt()` sends graceful message but doesn't guarantee stop 2. No visual feedback — cancelled message only persisted in catch block, which never ran on interrupt ## Solution Replace `query.interrupt()` (graceful) with `query.close()` (force-kill subprocess + children): - Terminates entire process tree, no need to track individual sub-agent tasks - Synchronous operation, immediate subprocess death Add `cancelledByUser` flag to SessionState: - Set by `handleCancel` before close - Checked in post-loop path to save cancelled message + send Tauri event - Checked in catch block for early return (avoid duplicate error handling) ## Changes **Sidecar (agent cancellation):** - `handleCancel`: Use `close()` instead of `interrupt()`, set `cancelledByUser=true` - Post-loop: When `cancelledByUser`, save cancelled message to DB with `stop_reason: "cancelled"` envelope format - Catch block: Return early if `cancelledByUser` to avoid error handling path - Add TIMING instrumentation throughout for observability **Backend (cleanup dead code):** - Remove `cancelled_at` column updates from POST `/sessions/:id/stop` - Delete unused `getLatestUserMessage` function **Frontend:** - Remove `window.confirm()` blocking confirmation dialog - In AssistantTurn: When cancelled, strip sentinel message from array, put all real messages in hiddenMessages, render warning-styled "Response stopped" badge instead of summary message **Networking/IPC:** - Add RPC logging to understand cancel flow - Socket.rs: Log cancel events ## Testing - `bun run test:sidecar` — all 198 tests pass - `bun run test:backend` — all tests pass - Manual: Start session → click Stop → agent stops immediately, "Response stopped" appears, session idle - Manual: Restart app → cancelled session persists, shows "Response stopped" badge Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com> * fix: address PR review — redact prompts, fix scope bug, safe UTF-8 truncation Addresses 3 actionable findings from CodeRabbit review: 1. **Redact raw prompts from logs** (comments #2 + #5): Replace `prompt.slice(0, 80)` with `promptLength` in timing logs in both claude-handler.ts and index.ts. User prompts were being written to /tmp log files — now only the length is logged. 2. **Fix messageCount ReferenceError** (comment #3): Move `let messageCount` declaration from inside the `try` block to before it. In JS, `let` inside `try {}` is NOT accessible in `catch {}` — they are separate block scopes. This caused a ReferenceError in the catch block, masking the actual error. 3. **Safe UTF-8 truncation in Rust** (comment #6): Replace `&error[..error.len().min(80)]` with `error.chars().take(80).collect::<String>()` in socket.rs. Byte-index slicing can panic on multibyte UTF-8 boundaries. (session_id truncation left as-is — UUIDv7 is guaranteed ASCII.) Dismissed as not actionable: - Comment #4 (error redaction in session-writer): Error messages are already sanitized by classifyError(). Redacting would lose debuggability for zero security gain. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com> * fix(test): add mockDbAll to session-writer test mock reconcileStuckSessions now calls .prepare().all() for diagnostic logging before the UPDATE. The test mock only had run/get methods, causing .all() to be undefined and the function to hit the catch block. This is a test mock gap, not a production bug. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
zvadaadam
added a commit
that referenced
this pull request
Mar 12, 2026
- #3: Log query:invalidate emit errors in backend.rs (was silent `let _ =`) - #5: Guard malformed payload in useQueryInvalidation with `?.` fallback - #6: Update CLAUDE.md — useSession is event-driven, not polled - #7: Move misplaced ChatInsert JSDoc to correct position above its schema Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
zvadaadam
added a commit
that referenced
this pull request
Mar 12, 2026
#5) - #2: Clear error_message/error_category in writeUserMessage() to align with sidecar's saveUserMessage() — prevents stale error fields on retry - #5: Tighten QueryInvalidateSchema resources to z.enum(QUERY_RESOURCES) instead of z.string() — catches misspelled resource names at runtime Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
zvadaadam
added a commit
that referenced
this pull request
Mar 12, 2026
* feat(events): centralized event catalog with Zod runtime validation Implement a single source of truth for all Tauri/sidecar events with compile-time type safety and runtime validation: - Add shared/events.ts with 17 event constants, 13 Zod schemas, 3 domain arrays (QUERY_RESOURCES, MUTATION_NAMES, SIDECAR_NOTIFY_EVENTS), and AppEventMap/AppEventName types - Wire Zod validation into listen() wrapper — validates payloads at Rust→TS boundary, logs drift, still delivers on failure - Fix 3 raw string emit() calls to use BROWSER_WORKSPACE_CHANGE and CHAT_INSERT constants - Consolidate message-writing to writeUserMessage() service to eliminate duplication - Remove web polling fallback code (getWorkspacesByRepoRefetchInterval, getStatsRefetchInterval) — desktop-only app - Fix type narrowing for INVALIDATION_MAP in notify route - Tighten ChatInsertSchema element variant with proper InspectElement Zod schema - Add 32 tests: events schema validation (19), listen() Zod validation (5), notify route (8) - All 286 backend + 131 frontend + 442 sidecar tests pass Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com> * docs(CLAUDE.md): document event catalog pattern and update polling guidance - Add Event Catalog section explaining shared/events.ts, AppEventMap, Zod validation, createListenerGroup, and how to add new events - Update request volume table to reflect event-driven invalidation (workspaces/stats no longer polled) - Update polling discipline to reference push-first invalidation pipeline - Update read-layer migration priorities (no longer referenced as "polled every 2s") Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address code review feedback (#3, #5, #6, #7) - #3: Log query:invalidate emit errors in backend.rs (was silent `let _ =`) - #5: Guard malformed payload in useQueryInvalidation with `?.` fallback - #6: Update CLAUDE.md — useSession is event-driven, not polled - #7: Move misplaced ChatInsert JSDoc to correct position above its schema Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: tighten QueryInvalidateSchema and clear error fields on retry (#2, #5) - #2: Clear error_message/error_category in writeUserMessage() to align with sidecar's saveUserMessage() — prevents stale error fields on retry - #5: Tighten QueryInvalidateSchema resources to z.enum(QUERY_RESOURCES) instead of z.string() — catches misspelled resource names at runtime Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
zvadaadam
added a commit
that referenced
this pull request
Mar 22, 2026
PairGatePage: - Remove last "relay" jargon from error message (#3) - PulsingDots respects prefers-reduced-motion via `reduced` prop (#4) - Add sr-only label, aria-invalid, aria-describedby, role=alert for a11y (#5) - Store success timeout in ref + cleanup on unmount (#6) - Cancel paste auto-submit timer on type and explicit submit (#8) AccessSection: - Use onOpenChangeRef to prevent stale closure in success setTimeout (#7) - Detect new devices by ID set comparison instead of count (#9) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 089f8af05d92
zvadaadam
added a commit
that referenced
this pull request
Mar 22, 2026
) * feat: redesign device pairing UX with two-word codes, QR dialog, and animated gate page Replaces WORD-NNNN pairing codes with two uppercase words (e.g. SOFT TIGER) from a 250-word list for better dictation and mobile typing. Consolidates the split two-field input into a single field with smart paste handling. Adds a centered dialog with QR code hero for desktop pairing flow, and redesigns the remote PairGatePage with three animated states (auto-connecting, success, manual entry). Codes are now reusable within their 15-minute TTL for multi-device sharing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: replace hardcoded color classes with semantic tokens Use bg-success, text-success, text-warning, bg-warning, bg-destructive instead of bg-emerald-500, text-emerald-500, text-amber-500, bg-red-400. These semantic tokens are already defined in global.css and adapt to light/dark mode via OKLCH values. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 9ffb5ff87432 * fix: update integration tests for two-word codes and reusable TTL The integration test still expected the old WORD-NNNN format regex and one-time-use code behavior. Updated to match two-word format and verify that codes are reusable within their TTL for multi-device sharing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 36c4f0b6f8fc * fix: address code review findings for pairing UX PairGatePage: - Remove last "relay" jargon from error message (#3) - PulsingDots respects prefers-reduced-motion via `reduced` prop (#4) - Add sr-only label, aria-invalid, aria-describedby, role=alert for a11y (#5) - Store success timeout in ref + cleanup on unmount (#6) - Cancel paste auto-submit timer on type and explicit submit (#8) AccessSection: - Use onOpenChangeRef to prevent stale closure in success setTimeout (#7) - Detect new devices by ID set comparison instead of count (#9) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 089f8af05d92 * fix: gate device success detection on query resolution Pass isDevicesLoaded (from devicesQuery.isFetched) to ConnectDeviceDialog. When the query hasn't resolved yet, just update the baseline IDs without triggering success detection. This prevents false positives when the dialog opens while devices are still loading ([] → existing devices). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: f34b15c1a77e * fix: differentiate QR placeholder states in connect dialog Show "Generating..." only when the relay URL is available but the code is still being created. Show "Waiting for connection..." when the relay itself hasn't connected yet (no accessUrl). Previously both states showed "Generating..." which was misleading. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 43c76b5ad359 * fix: await clipboard writes and use curated error messages - Await navigator.clipboard.writeText() and show error toast on failure instead of assuming success (all 3 clipboard call sites) - Replace raw backend error messages in toasts with curated user-facing copy ("Couldn't generate a code", "Couldn't remove device") - Remove unused getErrorMessage import Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 60d8c8e60b18 --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
zvadaadam
added a commit
that referenced
this pull request
Apr 2, 2026
…mentation Fixes 9 out of 10 code review issues (#1-5, #7, #9-10): - Remove dead Claude transformer accumulation (Issue #1) - Add graceful fallback for full_message column detection (Issue #2) - Use process.argv[1] for bundle-safe vendor path resolution (Issue #3) - Static import of execSync instead of dynamic require (Issue #4) - Add platform support guard instead of silently mapping to Linux (Issue #5) - Add null check on child.stdout before readline (Issue #7) - Add identity check in finally block to prevent race condition (Issue #9) - Capture thread_id for Codex message correlation (Issue #10) The in-place mutation in codex-adapter.ts (Issue #6) is verified safe because blocks are JSON-serialized before reaching the frontend, making the mutation internal to adapter lifecycle and invisible to consumers. All 193 sidecar tests pass. Zero type errors. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
zvadaadam
added a commit
that referenced
this pull request
Apr 2, 2026
- #3: Log query:invalidate emit errors in backend.rs (was silent `let _ =`) - #5: Guard malformed payload in useQueryInvalidation with `?.` fallback - #6: Update CLAUDE.md — useSession is event-driven, not polled - #7: Move misplaced ChatInsert JSDoc to correct position above its schema Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
zvadaadam
added a commit
that referenced
this pull request
Apr 2, 2026
#5) - #2: Clear error_message/error_category in writeUserMessage() to align with sidecar's saveUserMessage() — prevents stale error fields on retry - #5: Tighten QueryInvalidateSchema resources to z.enum(QUERY_RESOURCES) instead of z.string() — catches misspelled resource names at runtime Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
zvadaadam
added a commit
that referenced
this pull request
Apr 2, 2026
## Changes ### 1. Fix DevTools Expansion Bug (Tauri Native Layer) When clicking the DevTools button, the browser view would unexpectedly expand. Root cause: `_inspector.show()` docks by splitting WKWebView's superview, resizing the webview. After `detach()` moves inspector to floating window, frame isn't restored. **Fix:** Save WKWebView frame before show(), restore after detach(). - Modified: src-tauri/src/commands/webview.rs (lines 570-582) ### 2. Fix Cursor Animation Attribute Mismatch (Issue #1 - HIGH) Cursor visual effects (move, ripple, pin) were silently failing because they searched for `data-cursor-ref` but elements were marked with `data-hive-ref`. **Fix:** Replace all `data-cursor-ref` references with `data-hive-ref` to match the actual attribute used in inject-mode.ts. - Modified: src/features/browser/automation/visual-effects.ts - Line 18: JSDoc comment - Line 25: buildMoveCursorAndRippleJs() - Line 45: buildPinCursorJs() - Line 96: buildHighlightElementJs() ### 3. Add IME Composition Guard to MessageInput (Issue #2 - MEDIUM) CJK (Chinese, Japanese, Korean) users using Input Method Editors (IME) would lose their input when pressing Enter during composition, since the handler didn't check if composition was in progress. **Fix:** Add `!e.nativeEvent.isComposing` guard to Enter key handler to prevent message send during active IME composition. - Modified: src/features/session/ui/MessageInput.tsx (line 286) ### 4. Fix Element Selector Injection Exception Handling (Issue #3 - MEDIUM) When `verify_injection` script threw an exception, the catch block didn't return, causing exception object to fall through to `eval_browser_webview_with_result`, which would parse it as a result string instead of error. **Fix:** Add explicit `return;` in catch block to prevent fall-through. - Modified: src/features/browser/ui/BrowserTab.tsx (line 521) ### 5. Fix Selector Toggle State on Eval Failure (Issue #4 - MEDIUM) When element selector injection failed, `selectorActive` was set in the UI store but outside the try block, so errors during injection were silently swallowed. **Fix:** Move state update inside try block to fail-fast on injection errors. - Modified: src/features/browser/ui/BrowserTab.tsx (line 563) ### 6. Add Polling Guard to Inspect Event Drain (Issue #5 - MEDIUM) Inspect event drain was polling unconditionally every 200ms, consuming ~5 IPC calls/second unnecessarily. With multiple tabs this compounds quickly. **Fix:** Only poll when selector is active. Add `!tab.selectorActive` guard and add to dependency array. - Modified: src/features/browser/ui/BrowserTab.tsx (lines 396-458) ### 7. Replace if/else with ts-pattern (Issue #6 - LOW) Long if/else chain for event type dispatch violates CLAUDE.md requirement to use ts-pattern for discriminated union dispatch. **Fix:** Import ts-pattern and use `.with()` / `.otherwise()` for event type matching. - Modified: src/features/browser/ui/BrowserTab.tsx (lines 26, 427-444) ### 8. Fix Hover Animation Timing (Issue #7 - LOW) InspectedElementCard hover transition was `duration-150` instead of `duration-200`, violating CLAUDE.md animation standard of 200-300ms default duration. **Fix:** Update to `duration-200 ease` to match project animation guidelines. - Modified: src/features/session/ui/InspectedElementCard.tsx (line 48) ## Verification - TypeScript compilation: ✓ (pre-existing Lucide icon issue unrelated to changes) - All fixes verified and applied by parallel code-reviewer agents - Changes preserve existing functionality while fixing correctness issues - Performance improvement: Eliminates ~5 IPC calls/second from unconditional polling ## Impact - **Bug fixes:** 3 (DevTools expansion, cursor animations, IME composition) - **Correctness improvements:** 2 (exception handling, state sync) - **Performance:** Reduced IPC quota consumption from inspect event drain - **Code quality:** Full CLAUDE.md compliance (ts-pattern, animation timing) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
zvadaadam
added a commit
that referenced
this pull request
Apr 2, 2026
1. **inject-mode.ts: var() scan now covers padding/gap (#3)** Renamed colorProps → varScanProps, added padding/gap so getMatchedVarDeclarations captures var() tokens for spacing props. 2. **inject-mode.ts: font-weight fallback added (#10)** Added font-weight to the fallback conditional alongside font-size and border-radius. Filters out 'normal' (CSS default) as noise. 3. **BrowserPanel.tsx: DevTools state races with async invoke (#4)** Moved handleUpdateTab into .then() so devtoolsOpen only updates after the Rust command succeeds. Prevents UI desync on failure. 4. **BrowserTab.tsx: safeListen missing .catch() (#5)** Added .catch() to the listen() promise chain to prevent unhandled rejections if Tauri listener registration fails. 5. **BrowserTab.tsx: injectionFailed not cleared on success (#7)** Changed onUpdateTab to also set injectionFailed: false after successful injection, clearing stale error indicators. 6. **webview.rs: eprintln! gated for release builds (#8)** All 4 eprintln! calls in eval_browser_webview_with_result wrapped with cfg!(debug_assertions). Prevents leaking JS content, URLs, tokens, or page console data in production. 7. **webview.rs: DevTools error propagation (#9)** open/close_browser_devtools now use Arc<Mutex<Option<String>>> to propagate null-pointer errors out of with_webview closures. Frontend receives Err() instead of silent Ok(()) when DevTools unavailable. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
zvadaadam
added a commit
that referenced
this pull request
Apr 2, 2026
PairGatePage: - Remove last "relay" jargon from error message (#3) - PulsingDots respects prefers-reduced-motion via `reduced` prop (#4) - Add sr-only label, aria-invalid, aria-describedby, role=alert for a11y (#5) - Store success timeout in ref + cleanup on unmount (#6) - Cancel paste auto-submit timer on type and explicit submit (#8) AccessSection: - Use onOpenChangeRef to prevent stale closure in success setTimeout (#7) - Detect new devices by ID set comparison instead of count (#9) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 089f8af05d92
zvadaadam
added a commit
that referenced
this pull request
Apr 2, 2026
…uncation Addresses 3 actionable findings from CodeRabbit review: 1. **Redact raw prompts from logs** (comments #2 + #5): Replace `prompt.slice(0, 80)` with `promptLength` in timing logs in both claude-handler.ts and index.ts. User prompts were being written to /tmp log files — now only the length is logged. 2. **Fix messageCount ReferenceError** (comment #3): Move `let messageCount` declaration from inside the `try` block to before it. In JS, `let` inside `try {}` is NOT accessible in `catch {}` — they are separate block scopes. This caused a ReferenceError in the catch block, masking the actual error. 3. **Safe UTF-8 truncation in Rust** (comment #6): Replace `&error[..error.len().min(80)]` with `error.chars().take(80).collect::<String>()` in socket.rs. Byte-index slicing can panic on multibyte UTF-8 boundaries. (session_id truncation left as-is — UUIDv7 is guaranteed ASCII.) Dismissed as not actionable: - Comment #4 (error redaction in session-writer): Error messages are already sanitized by classifyError(). Redacting would lose debuggability for zero security gain. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
zvadaadam
added a commit
that referenced
this pull request
Apr 12, 2026
…safe parser in shared/messages/, added parts field to the shared Message type, and aligned backend persistence serialization with the shared schema — 14 new tests, all 459 backend + 371 agent-server tests passing.
zvadaadam
added a commit
that referenced
this pull request
Apr 12, 2026
* hmmm * gnhf #1: Created the complete unified message transformation layer: Part types in shared/messages/, Claude Code and Codex adapters in agent-server/messages/, with 29 passing tests covering text/reasoning/tool streaming, subagent tracking, compaction, and token usage — all without breaking any of the 347 existing tests. * gnhf #2: Wired the Claude adapter into the handler's streaming loop with dual-write: SDK events now produce unified Parts alongside legacy events, with new and canonical events flowing through EventBroadcaster to the backend. * gnhf #3: Wired the Codex SDK adapter into codex-handler.ts with dual-write: created a new CodexSdkTransformer that works directly with the SDK's ThreadEvent lifecycle (item.started/updated/completed), producing unified Parts alongside legacy events, with 23 new tests all passing (371 total agent-server tests). * gnhf #4: Added backend persistence for unified Parts: PartsAccumulator collects streaming parts, persistMessagePartsFinished attaches them to the last assistant message row via UPDATE (parts TEXT column), wired into the event handler with proper invalidation — 15 new tests, all 445 backend + 371 agent-server tests passing. * gnhf #5: Created MessagePartsEnvelope schema and parseMessageParts() safe parser in shared/messages/, added parts field to the shared Message type, and aligned backend persistence serialization with the shared schema — 14 new tests, all 459 backend + 371 agent-server tests passing. * gnhf #6: Wired parsed Parts into the frontend by adding partsMap (Map<messageId, MessagePartsEnvelope>) to useSessionWithMessages, SessionContext, and SessionProvider — components can now access typed Parts via useSession().partsMap for gradual adoption of the new rendering path, with TypeScript compilation and all 830 tests passing. * Unified message & parts architecture: agent-server events, backend persistence, DB schema Complete rewrite of the agent-server event model and backend persistence layer. The agent-server now emits 7 canonical lifecycle events that flow through the backend to the frontend, replacing the legacy dual-write system. ## Agent-Server (standalone event model) - 7 lifecycle events: turn.started → message.created → part.created/delta/done → message.done → turn.completed - 3 adapters (Claude Code, Codex CLI, Codex SDK) produce identical event types - Delta streaming: O(n) bandwidth — only new tokens sent, full text on done - Message boundaries: separate messages per API call (Claude: multi-message turns, Codex: single) - includePartialMessages: true for Claude SDK streaming - Turn boundary detection with turnVersion for session reuse - Debug CLI (bun run cli:agent-server) for testing both harnesses - PartEvent type in shared/agent-events.ts shared by agent-server and backend - Removed legacy sendMessage/sendError/sendStatusChanged from production code ## Backend (persistence + event forwarding) - Separate parts table (append-only, crash-safe) — one INSERT per part.done - message.created → INSERT message row; part.done → INSERT part row; message.done → UPDATE stop_reason - Zod schemas for all 7 events in AgentEventSchema - Part events forwarded to frontend via q:event WebSocket frames - Messages query JOINs parts via attachParts() - Legacy persistence functions deleted (persistAssistantMessage, persistToolResultMessage, etc.) - Backend debug CLI (bun run cli:backend) for end-to-end persistence testing - 12 integration tests verify event → DB pipeline ## Database Schema - New parts table: id, message_id, session_id, seq, type, data (JSON), tool_call_id, tool_name - messages table: added stop_reason, removed parts JSON column ## Tests 363 agent-server + 513 backend = 876 tests passing Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: CI failures — TypeScript errors, ESLint, e2e test, integration test skip - SessionPanel: use empty string instead of null for synthetic message content - ToolPartBlock: use createElement() instead of direct call (ComponentClass compat) - E2E test: wait for canonical session.idle/session.error instead of legacy message/queryError - Integration tests: gracefully skip when better-sqlite3 is compiled for Electron ABI Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address code review findings — TS error, lifecycle gaps, streaming overlay - Remove stop_reason from synthetic Message (TS2353 fix) - Remove dead emitMessageResult from Codex turn.completed handler - Add message.done emission to finish() in both Codex adapters - Fix streaming overlay preferring stale DB parts over live deltas - Fix synthetic message seq ordering (incrementing offset) - Forward all args in prepare-commit-msg hook - Mark stale agent memory doc as resolved Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: update Codex E2E test to use canonical events instead of legacy The Codex E2E test was still waiting for legacy `message`/`queryError` notifications which are no longer emitted. Updated to wait for canonical `session.idle`/`session.error` events, matching the Claude E2E test. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: update agent-server event model memory — remove stale findings All previously documented issues have been resolved. Rewrote to reflect current architecture: canonical lifecycle events, backend persistence, known patterns. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
zvadaadam
added a commit
that referenced
this pull request
Apr 14, 2026
…olor), un-exported 3 internal-only symbols (RECENT_PROJECT_LIMIT, resolveGitProjectRoot, setLastOpenInAppId), and replaced duplicate timeAgo in AccessSection.tsx with shared formatTimeAgo — net reduction of 41 lines across 5 files with all 825 tests passing.
zvadaadam
added a commit
that referenced
this pull request
Apr 14, 2026
* gnhf #1: Extracted two helper methods in claude-adapter.ts (closeActiveParts, accumulateStreamDelta) to eliminate 5 repeated close-text/close-thinking call sites and 2 duplicate 25-line streaming delta handlers; also replaced a duplicate 15-line formatTime function in WorkspaceItem.tsx with the existing shared formatTimeAgo utility — net reduction of 49 lines with all 375 agent-server tests passing. * gnhf #2: Extracted duplicate workspace-grouping logic (85 lines across query-engine.ts and workspaces.ts route) into shared lib/workspace-grouping.ts, and consolidated duplicate parameter-reading helpers (readString/readNumber across query-engine.ts and commands.ts) into shared lib/query-params.ts — net reduction of 41 lines with all 825 tests passing. * gnhf #3: Removed unused parameter from cancellation functions across both agent handlers, extracted helper replacing 5 identical repo-lookup patterns in repos.ts, unified duplicate git progress push functions, and removed dead export — net reduction of 23 lines across 7 files with all 825 tests passing. * gnhf #4: Removed 5 dead exports/functions and un-exported 2 internal-only functions across 6 files — net reduction of 112 lines with all 825 tests passing. * gnhf #5: Removed 2 dead exported functions (getRepoInitials, getRepoColor), un-exported 3 internal-only symbols (RECENT_PROJECT_LIMIT, resolveGitProjectRoot, setLastOpenInAppId), and replaced duplicate timeAgo in AccessSection.tsx with shared formatTimeAgo — net reduction of 41 lines across 5 files with all 825 tests passing. * gnhf #6: Un-exported 14 dead Zod schema validators (6 from shared/events.ts, 8 from shared/agent-events.ts), un-exported 6 dead type aliases from shared/events.ts, and consolidated duplicate parseGitHubRepo function from gh.service.ts and deus-import.ts into shared/lib/github.ts — reducing public API surface by 20 exports and eliminating 1 duplicate function across 5 files with all 825 tests passing. * gnhf #7: Deleted 2 dead component files (OpenInDropdown 214 lines, EmptyState 39 lines), removed 3 dead API type definitions (ApiResponse, PaginatedResponse, WorkspaceQueryParams) from shared/types/api.ts, removed 4 dead session type aliases (SessionMessageEvent, SessionErrorEvent, SessionEnterPlanModeEvent, SessionStatusEvent) from shared/types/session.ts, and cleaned up 3 barrel re-export files — net reduction of 330 lines across 7 files with all 825 tests passing. * gnhf #8: Deleted 2 dead platform files (updates.ts 31 lines, listenerGroup.ts 38 lines), removed dead function from dialog.ts, removed the entire dead StatusChanged notification pipeline across 5 files (method, schema, type, constant, test builder, union), un-exported from electron barrel, and cleaned up 3 barrel re-export files — net reduction of 121 lines across 11 files with all 825 tests passing. * gnhf #9: Removed 3 dead query hooks (useStats, useUncommittedFiles, useLastTurnFiles) with their stub service methods and query keys, removed 4 dead type definitions (ChangedFilesResult, BranchInfo, PaginationParams, DevServer) and their barrel re-exports, un-exported internal-only connectToRelay function, and cleaned up 2 dead barrel re-exports (clearToken, ConnectionIllustration) — net reduction of 121 lines across 12 files with all 825 tests passing and clean tsc. * gnhf #10: Removed 6 dead visual effect builder functions (98 lines) from visual-effects.ts, un-exported 5 internal-only symbols (resolveClaudeDir, getAgentConfig, StatusPriority, StatusConfig, WorkflowStatusConfig), and removed dead barrel re-exports (createAgentEventHandler, AgentEventHandler) from agent/index.ts — net reduction of 98 lines across 5 files with all 825 tests passing and clean tsc. * gnhf #11: Consolidated inline path validation in files.ts to use shared resolveWorkspaceRelativePath from git.service.ts, and replaced 10 hand-rolled readString+throw param validation patterns in commands.ts with the existing requireParam utility — net reduction of 17 lines across 2 files with all 825 tests passing and clean tsc. * gnhf #12: Removed 211 lines of dead CSS from global.css: the entire glitch-swap effect system (6 classes, 6 @Keyframes, 2 media queries) and the empty tool-use-enter class — none referenced by any component, with all 825 tests passing. * gnhf #13: Deleted 2 dead component files (RepoGroup.tsx 102 lines, WorkspaceItem.tsx 123 lines) from repository/ui that were never rendered anywhere, removed 14 dead barrel re-exports across 4 barrel files (repository/ui, sidebar/ui, sidebar feature, session/ui/blocks), and cleaned up knip.json — net reduction of 242 lines across 7 files with all 825 tests passing and clean tsc. * gnhf #14: Deleted the dead agent-server/messages/index.ts barrel (30 lines), removed 4 dead type exports from shared/messages/types.ts (ToolLocation, ToolOutputContent, PartType, PartTypeSchema), removed dead ToolResultMap type from chat-types.ts, and cleaned up 15 dead barrel re-exports across 4 barrel files (session/hooks, session/ui, workspace/ui, shared/hooks) — net reduction of 51 lines across 8 files with all 825 tests passing and clean tsc. * gnhf #15: Removed 10 dead type aliases from shared/agent-events.ts, un-exported 2 dead Zod schemas from shared/enums.ts, removed the dead useWindowFocus hook and its useSyncExternalStore infrastructure (50 lines) from useWindowFocus.ts, and removed dead BaseToolRendererProps barrel re-export — net reduction of 62 lines across 4 files with all 825 tests passing and clean tsc. * refactor: extract backend helpers and fix typecheck * Address CodeRabbit review feedback * Fix Claude adapter message handoff
zvadaadam
added a commit
that referenced
this pull request
Apr 18, 2026
17 inline comments from the bot — addressing all of them. Brief
mapping (file → comment id):
CRITICAL
package.json#62 Add missing zustand dep (was hoisted to root, would
fail bun install --frozen-lockfile in CI).
MAJOR
agentic-app.json#33 UI/MCP URLs use 127.0.0.1 instead of localhost so
host IDEs don't trip over IPv4/IPv6 happy-eyeballs
ambiguity. serve.ts also defaults --host to 0.0.0.0
(matches the server's own default), and the printed
probe URL collapses 0.0.0.0 → 127.0.0.1.
serve.ts#33 Compiled-layout candidate path was off-by-one
(..server vs ../..server). Fixed.
serve.ts#65 Browser only opens after /health responds 200; on
startup failure, no dead tab.
serve.ts#72 Propagate child exit code; non-zero exits return
{success:false} with the code.
e2e-server.ts#173 ChildProcess.killed is set when the signal is sent,
not when the process exits — using it as a kill-fallback
check would race a SIGTERM-ignoring child. Switched
to a tracked `exited` flag + clearTimeout on exit.
Sidebar.tsx#17 Refresh refs whenever streamUdid changes (sim swap),
not only when refs.length === 0. Stops stale refs
from a previous sim from lingering.
styles.css#85 Added --on-accent / --stage-bg / --frame-bg /
--canvas-bg / --shadow-* tokens; removed every
literal `white`, `#06060a`, `#1a1a22`, `#000`, and
`rgba(...)` from the stylesheet body.
state.ts#72 persist() now catches mkdir/writeFile failures
inside the chained promise so a single transient
error can't permanently brick the writeChain.
tools.ts#562 stream_logs wires onError + onExit to delete the
handle from logHandles, so failed/exited streams
don't leak.
MINOR
design-doc#57,#65 Architecture diagram said "TanStack Start on Nitro"
but the locked stack (decision #5) is "Hono on
Bun.serve" / "Vite + React". Aligned.
README.md#88 Added `text` language tag to the architecture fenced
code block (markdownlint MD040).
project-info.ts#43 detectKind() throws XcodebuildError on inputs that
aren't .xcodeproj or .xcworkspace, instead of
silently treating them as .xcodeproj.
xcodebuild.ts#159 Removed the synthesized appPath from BuildResult —
it was a guess based on `scheme.app` that's wrong
whenever target product name differs. Callers
should use resolveAppPath() (already used by the
`run` composite tool).
TopBar.tsx#15 onBoot pushes a failed tool-event into the activity
store on api.boot() failure, so the toast surfaces
the error.
sim-store.ts#67 setPinned checks res.success — only commits the
pinnedUdid change on success, otherwise stores
error state.
styles.css#345 Replaced deprecated `word-break: break-word` with
`overflow-wrap: anywhere` (modern equivalent).
Side effects:
scripts/*.ts Added `export {}` to the three smoke scripts so
tsc treats them as modules, not global scripts
(was causing duplicate-symbol errors after a
second smoke script joined the project).
Verified:
- 86/86 unit tests pass
- typecheck clean
- bun scripts/e2e-server.ts passes against a real iPhone 17 Pro
(17 tool-events, 97 tool-log frames, 0 failures)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
7 tasks
zvadaadam
added a commit
that referenced
this pull request
Apr 18, 2026
All 5 are doc-only fixes in docs/device-use-v2-design.md:
- Storage path duplication — `state.json` lives at
`{storage.workspace}/state.json`, not `{storage.workspace}/.device-use/state.json`
(the `.device-use` segment is already inside `storage.workspace`).
Fixed in 3 places (table row #12, backend wiring section).
- Architecture fence missing language — added `text` (markdownlint MD040).
- /ws bullet overstated CLI visibility — clarified that only actions
routed through the server (REST, MCP, viewer) populate the activity
stream; standalone CLI invocations bypass it (peer model).
- Phase 1 stack text was stale — said "Add TanStack Start" + "Nitro"
but the locked stack (decision #5) is Hono on Bun.serve + Vite.
Bullet rewritten to match what was actually built.
- Out-of-scope note about agentic-app.json was outdated — the manifest
IS in this PR; the next PR is the Deus host that consumes it. Reworded
+ retitled the manifest section to match.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
zvadaadam
added a commit
that referenced
this pull request
Apr 18, 2026
* feat(device-use): Phase 1 — scaffold v2 server + frontend
Adds the foundation for device-use v2 as a standalone product: a Hono
server on Bun.serve that will later host /mcp, /ws, /api/*, and a
React viewer at /. Dev mode proxies to Vite for HMR; prod serves the
built bundle from dist/frontend.
- Hono server at packages/device-use/src/server/index.ts with /health
- React scaffold at packages/device-use/src/frontend/ (main/App/index.html)
- Vite config rooted at src/frontend, outputs to dist/frontend
- tsconfig: JSX + DOM libs
- scripts: bun run dev (server), dev:frontend (vite), build, start
Also lands docs/device-use-v2-design.md — the full design record
(15 locked architectural decisions, phased implementation plan,
draft AAP manifest for later).
Part of the AAP (Agentic Apps Protocol) effort. This PR is the
device-use standalone refactor; AAP protocol itself is a separate PR.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(device-use): Phase 2 — expand engine with xcodebuild, build wrapper, logs
Adds three new engine primitives needed for the v2 "build + run + observe"
flow, each injectable for unit tests:
- project-info.ts — parses `xcodebuild -list -json` for an .xcodeproj or
.xcworkspace; returns schemes / targets / configurations. Used by the
agent (and the viewer's scheme picker) to know what to build.
- xcodebuild.ts — `build()` wraps `xcodebuild build` with spawn so the
stdout/stderr line stream can be piped into tool-log events (for the
viewer's logs drawer and the agent's per-build trace). Supports
AbortSignal so stop-app can cancel an in-flight build. Predicts the
built .app path when a derivedDataPath is provided; `resolveAppPath()`
is the fallback that queries -showBuildSettings -json.
- logs.ts — `streamLogs()` wraps `simctl spawn <udid> log stream` with
injectable spawner. Supports bundleId + pid filtering via --predicate,
and a stop() handle for graceful teardown.
All three follow the existing engine pattern (executor/spawner as first
arg, typed errors extending DeviceUseError). 18 new unit tests — all
using fake executors/spawners, zero real subprocess deps.
Exit: bun test passes 72/72; typecheck clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(device-use): Phase 3 — REST API, MCP HTTP endpoint, WS event bus
Wires all 23 MCP tools behind a single invoker that emits tool-event
frames over WebSocket. The same invokeTool path handles REST, MCP, and
WS calls — one engine path, one trace, zero duplicated behavior.
New modules:
- state.ts — persistent JSON at {storage}/state.json (pinned UDID,
active project/scheme). Write-serialized to avoid clobbers.
- events.ts — EventBus with 200-event ring buffer + subscriber set.
Emits ToolEvent + ToolLog; reuses MCP tool-call shape (no parallel type).
- invoker.ts — central dispatcher. Every MCP/REST/WS tool call goes
through invokeTool; emits started/completed/failed with a shared id.
- tools.ts — 23 tool definitions (Zod-schema'd handlers). Each handler
can use ctx.executor / ctx.state / ctx.stream / ctx.refMap / ctx.events.
- mcp.ts — stateless MCP HTTP handler. Creates a fresh Server+Transport
per request (SDK requirement for stateless mode) and routes tool calls
through invokeTool.
- stream.ts — StreamManager owns the long-lived simbridge --stream
subprocess; exposes proxyStream() for /stream.mjpeg passthrough.
Server wires them together:
- GET /health — liveness
- GET /api/state — persisted state
- GET /api/tools — all tools + JSON schemas
- POST /api/tools/:name — invoke a tool (REST)
- GET /api/events — recent tool-event history
- GET /stream.mjpeg — MJPEG passthrough
- GET /api/stream — stream status
- ALL /mcp — MCP HTTP transport
- WS /ws — tool-event + tool-log subscribe + invoke
Deps: @modelcontextprotocol/sdk, zod-to-json-schema.
Exit verified end-to-end:
- 86 tests pass (14 new integration tests)
- typecheck clean
- curl /api/tools → 23-tool JSON
- curl /mcp tools/list + tools/call → proper SSE responses
- scripts/ws-smoke.ts → WS receives started+completed tool-event after
REST invoke, correlation id matches
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(device-use): Phase 4 — React viewer (TopBar, DeviceFrame, Sidebar, logs)
Builds out the standalone UI served at /. Works entirely through the
HTTP + WS endpoints landed in Phase 3 — no direct engine imports
from the frontend.
Components:
- TopBar: sim picker, project path input, scheme dropdown,
▶ Run button with disabled/building/failed states
- DeviceFrame: phone bezel + MJPEG stream from /stream.mjpeg;
placeholder when no sim is pinned or sim not booted;
ripple animation dispatched from tap tool-events via a Zustand
subscription (not a useEffect setState)
- Sidebar: snapshot-on-demand with clickable @ref list
(each taps the element) + live activity feed of recent tool-events
- LogsDrawer: auto-scrolled tail of tool-log frames
- Toasts: one-liner on build/install/launch completion or any failure
Stores (Zustand):
- sim-store — sims list, pinned UDID, stream info
- project-store — active project path, scheme, configuration,
build status
- activity-store — tool-event ring buffer + toast dispatch
- logs-store — tool-log ring buffer (500 lines)
Infra:
- lib/ws.ts — WS client hook with exponential reconnect.
Routes tool-event → activity-store, tool-log → logs-store.
- lib/api.ts — typed fetch wrappers around /api/tools/*.
- styles.css — minimal dark theme using the same color tokens
as the Deus IDE. No Tailwind (keeps deps light).
Exit verified:
- typecheck clean, eslint clean
- vite build succeeds (202KB JS, 5KB CSS, 43 modules)
- Hono→Vite proxy serves the page via /
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(device-use): Phase 5 — CLI cleanup (drop SDK, drop stream CLI, add serve)
The legacy SDK (fluent session builder) and CLI `stream` subcommand
were built for a stateless-CLI world. v2 replaces them with the
long-lived server, MCP endpoint, and /stream.mjpeg passthrough —
so these surfaces are dead weight.
Deleted:
- src/sdk/ (fluent `session()` builder + apps helpers)
- src/cli/stream/ (manager + viewer-html generator)
- src/cli/commands/stream.ts (CLI stream enable/disable/status)
- scripts/sdk-smoke.ts
Added:
- src/cli/commands/serve.ts — `device-use serve [--port N] [--host] [--open]`
spawns bun on the server module and inherits stdio. Works in dev
(source path) and in bundled form.
Updated:
- src/cli/index.ts — drop stream import, add serve
- scripts/build-ts.ts — drop sdk build entry; now produces
dist/cli.js + dist/engine.js
- package.json — description updated; main/exports
point at engine (primary programmatic
entry); bin unchanged
CLI surface (preserved): list, boot, shutdown, open, session, snapshot,
query, tap, swipe, type, fill, screenshot, wait-for, open-url, apps,
appstate, launch, terminate, permission, doctor, install, serve.
Exit verified:
- typecheck clean
- 86 tests pass
- `bun run cli help` lists all commands including `serve`
- `bun run cli serve --port 3101` starts server, /health → 200
- `bun run build:ts` produces dist/cli.js + dist/engine.js cleanly
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(device-use): Phase 6 — docs, AAP manifest, composite run tool
Final phase: makes device-use v2 AAP-ready and documents the shape.
- agentic-app.json — manifest at the package root. Host IDEs (Deus,
MCP-speaking IDEs) read this to launch device-use, probe readiness
via /health, embed the viewer at /, and wire agent tool calls into
/mcp. Uses the schema shape agreed in docs/device-use-v2-design.md.
- run tool (composite) — chains build → resolveAppPath → install →
launch_app in one call. Reads CFBundleIdentifier from the built
Info.plist when bundleId isn't provided. Total MCP tools now 24.
The Viewer's ▶ Run button calls build + install + launch as
separate steps today; future phases can switch to this composite
for a cleaner progress story.
- README rewritten — new architecture, how to run the server, MCP
endpoint config for Claude Desktop, full tool list.
- AGENTS.md rewritten — new layout, how to add a tool (invokeTool
path), how to add a CLI command, event shape, hard rules ("never
bypass invokeTool", "stateless CLI stays stateless", etc.).
- .gitignore — adds .device-use/ (server's per-workspace state dir).
End-to-end verified:
- typecheck clean
- 86 tests pass
- curl /health → {ok:true}
- curl /api/tools → 24 tools
- curl /mcp tools/list → 24 tools
- set_active_simulator persists via state.json
- WebSocket streams started+completed tool-events with correlation id
- Pre-flight Info.plist-read in run tool handles bundleId resolution
device-use v2 is now the standalone product the AAP will later consume.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(device-use): add end-to-end server harness against real simulator
scripts/e2e-server.ts spawns a fresh device-use server on port 3199,
subscribes to /ws, exercises the full data flow, and asserts event
correlation across REST, MCP, and WebSocket paths. Runs in ~25s when
xcodebuild has a warm DerivedData cache.
Covers:
1. Server lifecycle — spawn + /health probe + graceful SIGTERM.
2. Tool registry — REST /api/tools reports 24 tools.
3. MCP HTTP transport — initialize + tools/list + tools/call(get_state)
over JSON-RPC-over-SSE with per-request stateless transport.
4. Simulator lifecycle — list_devices, set_active_simulator,
boot (Shutdown → Booted).
5. Project introspection — get_project_info parses xcodebuild -list -json
and finds the TestApp scheme.
6. Composite run tool — build → resolveAppPath → install → auto-resolve
bundleId from Info.plist → launch. End-to-end in one call.
7. Snapshot — a11y tree walk, @refs assigned to interactive nodes.
8. Tap — ref-based tap through simbridge.
9. State persistence — state.json round-trip verified on disk AND
via get_state tool.
10. WS event correlation — every tool emits paired started+completed
frames with matching ids; build streams tool-log frames during
xcodebuild.
First successful run:
24 tools · 19 tool-events · 273 tool-log frames · 0 failures · 25s
Future regression: `bun scripts/e2e-server.ts` from the package dir.
Requires Xcode + an iPhone 17 Pro simulator available.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(device-use): bind server to 0.0.0.0 so IPv6 orphans can't shadow us
The server previously bound to 127.0.0.1 only. On a machine with a
stale IPv6 listener on the same port (e.g. an orphaned sim-helper
from agent-simulator — observed in the wild, PPID=1), browsers
resolving `localhost` to ::1 first would connect to the wrong server
and render the stale UI.
Binding to 0.0.0.0 (all IPv4 interfaces) means happy-eyeballs falls
back to 127.0.0.1 cleanly when ::1 has no listener, and any IPv6
orphan on the same port becomes harmless rather than a hijacker.
Root cause, not a workaround: the orphan was a real bug in older
tooling; this change makes us robust against that class of collision.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(device-use): boot + set_active_simulator now start the MJPEG stream
The stream subprocess (simbridge --stream) was only auto-starting at
server startup for a pre-pinned sim. Booting interactively via the UI
would pin + boot the sim but never start the stream — viewer stuck on
"Waiting for stream…" even though everything else worked (snapshot,
activity feed, etc.).
Fix:
- boot tool — after simctl boot succeeds, start the stream for that
UDID. Failures are logged but don't fail the boot (CLI callers can
still boot without needing a stream).
- set_active_simulator tool — if the newly-pinned sim is already
booted, start the stream too. Covers the "open viewer after sim
is already booted externally" path.
Both return the new stream info in their result so the UI doesn't
need to wait for the 10s refresh tick to pick it up.
Verified against a real sim: restart server → pinned UDID auto-boots
and streams on an assigned port → /stream.mjpeg returns
200 multipart/x-mixed-replace with frames.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(device-use): interactive tap + swipe on the stream
Mouse events on the MJPEG image now drive the simulator, not just
watch it. Click → tap(x, y). Drag → swipe(from, to) with an 8pt
threshold to distinguish the two. Cursor turns crosshair when a
clickable stream is loaded.
To map CSS pixels → simulator points, StreamManager.start() now
probes the upstream /config on ready and computes logical point
dimensions (assuming the standard 3x retina scale). StreamInfo
gains `size: { pxW, pxH, ptW, ptH }`, exposed via /api/stream.
The DeviceFrame multiplies the click's fractional position by
ptW/ptH to get point coords for the tap/swipe tools.
Ripple animations now use the dynamic ptW from streamInfo instead
of the previous hardcoded 402pt estimate, so they land on the
right spot on any device.
Every interaction routes through invokeTool → same activity feed
entries, same tool-event broadcast, same MCP trace. Zero new
invocation paths.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(device-use): use canvas + binary WS for interactive stream
Clicks on the MJPEG <img> silently did nothing — browsers don't fire
mouse events reliably on images that swap frames continuously. The
original Deus simulator viewer avoided this by rendering MJPEG frames
into a <canvas> and sending touch events as binary WebSocket frames
to simbridge's /ws (0x03 prefix + JSON payload, begin/move/end
lifecycle). Adopting that proven pattern.
Server:
- /sim-input WebSocket endpoint. Opens an upstream WS to simbridge's
/ws for the active stream and forwards binary frames one-way
(browser → simbridge). Buffers messages until upstream handshake
completes. Closes upstream when client disconnects.
- Upgrade flow attaches { kind: "events" | "sim-input", wsUrl? } to
the Bun WS data field so a single websocket handler can multiplex.
Frontend:
- DeviceFrame now renders the stream into a <canvas>. A requestAnimation
Frame loop drives ctx.drawImage(img, 0, 0) from an off-screen Image
pointed at /stream.mjpeg. Canvas fires mouse events reliably even
while frames are updating.
- Mouse events encode 0x03-prefixed TouchEventPayload (begin on
mousedown, move on drag, end on mouseup) and send normalized 0..1
coordinates via the /sim-input WS.
- Window-level mouseup catches drags that release outside the canvas.
- Ripples fire locally on mousedown — immediate UI feedback without
waiting for a server round-trip.
Verified end-to-end: tap at Maps' center (0.27, 0.20) → Maps app
launched with "Allow Location" permission dialog. scripts/touch-smoke.ts
exercises the full path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(device-use): auto-refresh the elements sidebar after each canvas tap
User reported the refs sidebar showed stale elements: the list still
referenced the home screen after they'd opened Maps. Fix: every canvas
mouseup schedules a snapshot refresh (debounced 500ms, collapses rapid
taps into one call). The sidebar also auto-snapshots the moment the
stream first goes live, and shows the foreground app name next to the
"elements" header so the user knows what they're looking at.
- refs-store.ts — new Zustand store with refs + foreground + loading,
a refresh() action, and a scheduleRefresh(delayMs) debouncer
- DeviceFrame — calls scheduleRefresh on canvas mouseup AND on the
window-level mouseup (drag-release-off-canvas path)
- Sidebar — consumes the refs store, auto-snapshots on first stream
connect, re-snapshots after any ref tap (sidebar → canvas tap →
screen changes → refs stale otherwise)
Activity feed also gets clearer: users can see which tap was theirs
(REST via api.tap from ref clicks) vs. which came from the direct WS
/sim-input path (those bypass invokeTool and don't show in activity).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(device-use): ref-based tap uses tapEntry (id → label → coords fallback)
Ref taps were hitting the bounding-box center, which misses the hit
target for elements like switches/checkboxes where the tappable area
is not centered on the a11y frame (e.g. Maps' "Traffic" checkbox —
the a11y node's center is on the label, the switch is to the right).
Fix: use the engine's existing tapEntry helper, which tries in order:
1. tapById(identifier) — simbridge scans the a11y tree and taps the
matching element's natural hit target
2. tapByLabel(label) — same, by accessibility label
3. tap(x, y) — coordinate fallback
This matches what the CLI's `tap @eN` command already does. Ref taps
in the viewer sidebar now toggle switches, check checkboxes, and
activate controls whose hit target differs from the a11y bounds.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(device-use): hardware-button row + context-menu suppression
Adds a row of hardware buttons (Home / Lock / Vol+ / Vol−) below the
phone. Clicks send 0x04 + JSON{button} frames to simbridge's /ws via
the existing /sim-input proxy — same transport as touch, so they're
instant and don't round-trip through invokeTool.
Also:
- Cmd+H / Cmd+L keyboard shortcuts for Home / Lock (respects focus;
ignores keys while typing in inputs)
- Canvas right-click now preventDefault's (no more browser context
menu popping over the stream)
- Button press triggers the same debounced refs-snapshot refresh that
canvas taps do, so the sidebar stays in sync after going home
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(device-use): post-cleanup — remove dead code + tighten server
Mechanical cleanup after the v2 refactor, no behavior changes. 86
tests + e2e-server (18 tool events, 97 tool-log frames) still pass.
Server:
- EventBus now owns `logHandles: Map<string, LogHandle>` directly.
stream_logs / stop_logs drop the `ctx.events as unknown as …` cast
and the `_logHandles ??= new Map()` dance.
- New `toolInputSchema()` helper in tools.ts wraps the zod-to-json-
schema call with its `as any` cast. /api/tools and /mcp tools/list
both use it; removes the duplicated expression and two casts.
- WS_SUBSCRIBERS map collapsed from `{unsubscribe, ws: unknown}` to
the unsubscribe function directly. The `ws: unknown` field was
never read.
- Dropped the `void buf;` no-op in stream.ts — the simbridge stderr
handler just discards the parameter.
Frontend:
- snapshot tool now returns an explicit `foreground: string | null`
(the root Application node's label). Frontend reads it as a named
field instead of digging into `tree[0].label`.
- snapshot call in api.ts stops requesting interactiveOnly:false
(it was inflating the sidebar with non-clickable static text).
Server default of `interactiveOnly: true` is what the UI wants.
- project-store: BuildStatus type trimmed from 6 states to 4
(idle|running|done|failed). The old "building|installing|launching"
sub-states were dead — nothing set them.
- TopBar's Run button wired to `api.run()` (the composite: build →
install → launch) instead of `api.build()`. "▶ run" actually runs.
- DeviceFrame: extracted `normalize(clientX, clientY)` helper so the
canvas mouseup and window-level mouseup share one coordinate path.
- Every scheduleRefresh() callsite dropped its explicit 500ms arg;
that's now the store's default.
CLI:
- `device-use serve --open` now polls /health (up to 6s) before
opening the browser, replacing the 1500ms blind sleep.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(device-use): drop Vol+/Vol− buttons — simbridge doesn't map them
simbridge's SimAccessibilityBridge.sendButton only dispatches home and
lock; volumeup/volumedown payloads reach the handler but are no-ops.
Keeping the UI honest by removing the two buttons that don't do
anything. If volume control comes back later, it'll need a sendButton
case in Swift first.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(device-use): stop the 10s blink — use primitive deps in DeviceFrame
sim-store refreshes /api/stream every 10s and writes a fresh
streamInfo object into state. Both effects in DeviceFrame (the MJPEG
canvas render loop and the /sim-input WebSocket) depended on the
whole streamInfo object, so each poll tore down the canvas, cleared
it, recreated the Image element, and reconnected the WS. Result: a
visible black flash on the phone every 10 seconds and a one-frame
gap in touch delivery if you happened to be mid-drag.
Fix: depend on primitive identity — streamUdid (and streamPort for
the WS). Same UDID/port → same effect instance, no teardown. If the
underlying sim actually changes UDID (user switches sim), both
effects re-fire correctly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(device-use): stop the blink at the source — sim-store preserves identity
The first stab at the 10s blink fixed DeviceFrame's effect deps but
missed two other places that still churned references on every poll:
1. sim-store.refresh() set streamInfo / sims / pinnedUdid
unconditionally, so every 10s each of those fields got a new
reference even when the content was unchanged. Anything subscribed
to them via `useSimStore()` re-rendered.
2. Sidebar's "take an initial snapshot on first stream connect"
effect depended on the full streamInfo object; every poll
re-fired the effect's guard check.
Real fix — at the store, not the consumers:
- sim-store now compares new vs. previous stream/sims/pinnedUdid
and reuses the old reference when content matches. No downstream
sees a spurious update.
- Sidebar effect now depends on `streamInfo?.udid` (primitive)
instead of the object, matching DeviceFrame's pattern. Even if
we later regress #1, #2 is independently safe.
Served bundle: index-Cre9C4oH.js.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(device-use): keyboard input + CI workflow + long-press verified
Three changes closing the last gaps before the PR opens.
1. Keyboard input on the viewer — keystrokes while focus isn't on
another input get batched into a 150ms idle buffer and flushed via
the type_text tool. Enter flushes with submit=true (presses Return
after). Backspace trims the local buffer. Modifier keys (Cmd/Ctrl)
are preserved for the existing hardware-button shortcuts
(Cmd+H / Cmd+L). Buffering collapses fast typing like "hello"
from 5 sequential simbridge subprocesses into one.
2. Long-press verified — scripts/longpress-smoke.ts holds begin for
1500ms, then releases end, and asserts the iOS context menu
appears (screen refs grow, including the "Dismiss context menu"
action). Passes against a real iPhone simulator. The architecture
already supported it: our /sim-input WS forwards begin+end raw to
simbridge, which keeps the touch down for whatever interval the
client chooses. No code needed — just proof.
3. GitHub Actions workflow for nightly e2e on macos-15. Picks any
available iPhone, boots it, runs scripts/e2e-server.ts end-to-end
(spawn → /health → WS → REST + MCP → build → install → launch →
tap → state persistence). Schedule + manual dispatch only; we
don't run per-PR (macOS minutes cost 10×). The e2e script now
honors $E2E_SIM_UDID to target a specific sim without changing
its default behavior for local dev.
Verified end-to-end:
- type_text via REST: {success:true}
- longpress-smoke.ts: refs 3 → 7, iOS context menu opened
- touch-smoke.ts: still passes
- e2e-server.ts: full flow passes (18 tool-events, 97 tool-logs)
- 86/86 unit tests pass
- typecheck clean
Out of scope for this PR (noted in design doc as v1.1):
- wheel→scroll mapping
- pinch/rotation multi-touch
- CLI actions appearing in /ws activity feed (peer model stays pure)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(device-use): handoff section for the next PR (Deus AAP host)
Adds a "Handoff to next PR" block to docs/device-use-v2-design.md so
whoever picks up the Deus-side AAP integration has a working brief.
Covers:
- What to build in Deus (apps.service.ts, app-registrar.ts, lifecycle
tools, frontend launcher) with concrete file paths.
- What to delete after AAP works (~2.5k LOC of legacy simulator code
in apps/web/src/features/simulator/ + simulator-context.ts +
deus-tools/simulator.ts).
- 8 lessons device-use learned that should shape AAP host behavior:
requires validation, {port} substitution, mandatory health probe,
storage env var, child.on(exit) > polling, low-latency input
back-channel, runtime setMcpServers, framework-agnostic frontend.
- Things device-use deliberately deferred (wheel-scroll, CLI in WS,
multi-sim side-by-side) and where they belong.
- Open design questions for the next PR.
- A concrete first-commit slice for the Deus AAP PR.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(device-use): address CodeRabbit review on PR #249
17 inline comments from the bot — addressing all of them. Brief
mapping (file → comment id):
CRITICAL
package.json#62 Add missing zustand dep (was hoisted to root, would
fail bun install --frozen-lockfile in CI).
MAJOR
agentic-app.json#33 UI/MCP URLs use 127.0.0.1 instead of localhost so
host IDEs don't trip over IPv4/IPv6 happy-eyeballs
ambiguity. serve.ts also defaults --host to 0.0.0.0
(matches the server's own default), and the printed
probe URL collapses 0.0.0.0 → 127.0.0.1.
serve.ts#33 Compiled-layout candidate path was off-by-one
(..server vs ../..server). Fixed.
serve.ts#65 Browser only opens after /health responds 200; on
startup failure, no dead tab.
serve.ts#72 Propagate child exit code; non-zero exits return
{success:false} with the code.
e2e-server.ts#173 ChildProcess.killed is set when the signal is sent,
not when the process exits — using it as a kill-fallback
check would race a SIGTERM-ignoring child. Switched
to a tracked `exited` flag + clearTimeout on exit.
Sidebar.tsx#17 Refresh refs whenever streamUdid changes (sim swap),
not only when refs.length === 0. Stops stale refs
from a previous sim from lingering.
styles.css#85 Added --on-accent / --stage-bg / --frame-bg /
--canvas-bg / --shadow-* tokens; removed every
literal `white`, `#06060a`, `#1a1a22`, `#000`, and
`rgba(...)` from the stylesheet body.
state.ts#72 persist() now catches mkdir/writeFile failures
inside the chained promise so a single transient
error can't permanently brick the writeChain.
tools.ts#562 stream_logs wires onError + onExit to delete the
handle from logHandles, so failed/exited streams
don't leak.
MINOR
design-doc#57,#65 Architecture diagram said "TanStack Start on Nitro"
but the locked stack (decision #5) is "Hono on
Bun.serve" / "Vite + React". Aligned.
README.md#88 Added `text` language tag to the architecture fenced
code block (markdownlint MD040).
project-info.ts#43 detectKind() throws XcodebuildError on inputs that
aren't .xcodeproj or .xcworkspace, instead of
silently treating them as .xcodeproj.
xcodebuild.ts#159 Removed the synthesized appPath from BuildResult —
it was a guess based on `scheme.app` that's wrong
whenever target product name differs. Callers
should use resolveAppPath() (already used by the
`run` composite tool).
TopBar.tsx#15 onBoot pushes a failed tool-event into the activity
store on api.boot() failure, so the toast surfaces
the error.
sim-store.ts#67 setPinned checks res.success — only commits the
pinnedUdid change on success, otherwise stores
error state.
styles.css#345 Replaced deprecated `word-break: break-word` with
`overflow-wrap: anywhere` (modern equivalent).
Side effects:
scripts/*.ts Added `export {}` to the three smoke scripts so
tsc treats them as modules, not global scripts
(was causing duplicate-symbol errors after a
second smoke script joined the project).
Verified:
- 86/86 unit tests pass
- typecheck clean
- bun scripts/e2e-server.ts passes against a real iPhone 17 Pro
(17 tool-events, 97 tool-log frames, 0 failures)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(device-use): sync bun.lock with zustand workspace dep
The zustand dep was added to packages/device-use/package.json in
f14d701 but the workspace lockfile entry wasn't captured. This is
the missing one-line lockfile sync — `bun install --frozen-lockfile`
was already passing in CI because the root entry was up-to-date,
but the workspace block needed the addition for consistency.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(device-use): address CodeRabbit round 2 (3/4)
- design-doc#168 — removed stale "TanStack Start" reference from the
dev/prod modes row; replaced with the actual setup (Hono server with
--hot, separate Vite dev server, prod = bundled frontend + same
Hono server in NODE_ENV=production).
- sim-store#19 (sameStream) — extended comparison beyond udid + port
to include url and structural size comparison (pxW/pxH/ptW/ptH).
Stream metadata changes (e.g. orientation rotation altering size
while UDID/port stay fixed) now correctly notify subscribers.
- sim-store#30 (sameSims) — added runtime to the per-simulator
equality check. Runtime upgrades now propagate.
Disagreed with one suggestion (TanStack Query refactor for sim-store
fetches). Reasoning in the round summary on the PR — short version:
device-use is a standalone package whose frontend deliberately runs
on Zustand only, has no TanStack Query dependency today, and the
guideline cited applies to apps/web (Deus's main UI), not to
packaged sub-products. Adding TanStack Query for two REST reads
would inflate the bundle and introduce a paradigm split between the
WS event subscription and the polled data, with no real benefit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(device-use): clarify launch.ready manifest path in handoff section
CodeRabbit flagged that `launch.ready.http` was an inconsistent path
shorthand — the actual manifest schema is `launch.ready` with
`{type: "http", path: "/health"}`. Reworded to use the correct
shape so future readers don't assume a `launch.ready.http` key.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(device-use): address CodeRabbit round 4 (5 doc nits)
All 5 are doc-only fixes in docs/device-use-v2-design.md:
- Storage path duplication — `state.json` lives at
`{storage.workspace}/state.json`, not `{storage.workspace}/.device-use/state.json`
(the `.device-use` segment is already inside `storage.workspace`).
Fixed in 3 places (table row #12, backend wiring section).
- Architecture fence missing language — added `text` (markdownlint MD040).
- /ws bullet overstated CLI visibility — clarified that only actions
routed through the server (REST, MCP, viewer) populate the activity
stream; standalone CLI invocations bypass it (peer model).
- Phase 1 stack text was stale — said "Add TanStack Start" + "Nitro"
but the locked stack (decision #5) is Hono on Bun.serve + Vite.
Bullet rewritten to match what was actually built.
- Out-of-scope note about agentic-app.json was outdated — the manifest
IS in this PR; the next PR is the Deus host that consumes it. Reworded
+ retitled the manifest section to match.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(device-use): align manifest example with shipped agentic-app.json
CodeRabbit noticed the design-doc manifest example still showed
\`localhost\` URLs while the actual \`packages/device-use/agentic-app.json\`
uses \`127.0.0.1\` (the IPv4-safe form chosen during the round-1 fix
to avoid IPv4/IPv6 happy-eyeballs ambiguity). Aligned the example so
copy/paste from the doc doesn't regress the fix.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* style(device-use): adopt Deus design tokens (Jony-Ive dark theme)
The viewer now uses the same color/text/radius/font system that
apps/web/src/global.css defines for the Deus IDE itself, so embedding
device-use as an AAP app feels native instead of looking like a
separate skin.
Tokens mirrored 1:1 from Deus's dark theme:
- Backgrounds — six-tier hierarchy (--bg-base #0b0b0b → --bg-surface
→ --bg-elevated → --bg-raised → --bg-overlay → --bg-muted #2a2a2a).
Each step adds perceptible depth without needing borders.
- Text — five-level OKLCH grayscale (--text-primary 0.86 → 0.32),
generous gaps between levels.
- Accent — `oklch(0.78 0.09 345)` (Deus's "cool rose" primary), not
the previous purple. Used for the run button, status dot glow,
ref ids, and ripples.
- Semantic signals — equiluminant OKLCH (success / warning /
destructive all at L≈0.65-0.68), matches Deus exactly.
- Borders — `color-mix(in oklch, var(--foreground) 5%, transparent)`,
semi-transparent so they adapt to any surface.
- Radius scale — 2 / 4 / 6 / 8 / 10 / 12 / 16 / 20 / 24 px.
Device frame uses --radius-4xl (24px), canvas inside uses
--radius-3xl (20px), buttons --radius-md/lg.
- Typography — 13px base (matches Deus's dense IDE scale), system
-apple-system / SF Pro Text + SF Mono, negative letter-spacing
throughout (-0.005em / -0.015em on brand) for the "premium texture"
layer.
- Custom scrollbars themed to bg-raised → bg-overlay on hover.
JSX inline styles updated to reference the new tokens
(--accent-2 → --primary, --danger → --destructive, --dim →
--text-muted-foreground).
Class names unchanged — same DOM structure, just new visual tokens
underneath. Verified:
- 86/86 tests pass
- typecheck clean
- vite build: index-CKuhJYDJ.css (9.62 KB), index-BOvIo28T.js
- bun scripts/e2e-server.ts PASS — 18 tool-events, 97 tool-log
frames, 0 failures
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(device-use): collapsible logs drawer (collapsed by default)
The logs pane was eating 220px at the bottom even when empty, which
is most of the time. Now it's just a ~32px header row by default; the
phone + sidebar get the extra vertical space back.
- Click the header row to toggle expand/collapse.
- Chevron flips ▸ / ▾ to signal state.
- Auto-expands on any new log line — if you triggered a build or a
log stream, the drawer pops open for you. Collapsing is always
manual. Auto-expand uses a direct Zustand subscription (not a
useEffect setState) to satisfy react-hooks/set-state-in-effect.
- Clear button hidden when collapsed (no affordance without content).
- Grid row for logs changed from fixed 220px to `auto`; the
`.expanded` class pins the body at 220px when open.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(device-use): themed Select (Radix UI) — replaces native <select>
Native <select> popups use OS chrome (white on macOS), breaking the
dark Deus theme. Swapped for @radix-ui/react-select wrapped in a
small themed component — same primitive Deus's apps/web Select uses,
just styled with plain CSS against our token system (no Tailwind).
What's new:
- src/frontend/components/Select.tsx — minimal wrapper exposing a
plain {value, onValueChange, options, placeholder} API. Options
can be any ReactNode so the sim picker can render a state dot + a
dim runtime suffix inline.
- Styled trigger matches the other topbar inputs (bg-elevated, 1px
border, 6px radius, hover brightens to bg-raised, focus-visible
rings with --ring).
- Portaled popover uses --popover / --border / --shadow-elevated,
radius-xl; items have 6/10 padding, left checkmark indicator in
--primary, data-highlighted lifts to bg-elevated.
- Chevron rotates 180° on open (data-state="open").
- Smooth 180ms in / 150ms out with cubic-bezier matching apps/web.
TopBar wiring:
- Sim picker renders each option as `[dot] Name · runtime`, with the
dot green+glow when that sim is Booted.
- Scheme picker renders as a plain list.
- Empty "no simulators" case stays inline text (no dropdown needed).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(device-use): correct cache-control for prod static serving
Without explicit cache directives, browsers heuristically cache the
HTML shell and keep pointing at old Vite-hashed asset URLs across
deploys — user refreshes, sees a stale bundle. This was hitting the
dropdown-themed commit: the new bundle was built + served but the
browser's cached HTML still referenced the pre-Radix build.
Fix in the prod static-serving branch:
- /assets/* → `Cache-Control: public, max-age=31536000, immutable`
(Vite fingerprints these, so the URL changes whenever the content
does — safe to cache forever).
- Everything else (including the HTML shell) → `Cache-Control: no-cache`
(revalidate every load so new bundle URLs are picked up immediately).
Verified via `curl -D -`. HTML now sends `no-cache`, asset sends
`public, max-age=31536000, immutable`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(device-use): proper iPhone/iPad device frame (ported from apps/web)
The old frame was just a rounded rect with 12px padding and 40px
radius — didn't match iPhone proportions, had no dynamic island,
no camera hole, and no accurate bezel thickness. Ported the
GenericDeviceFrame shape from
apps/web/src/features/simulator/ui/DeviceFrame.tsx exactly:
- Aspect ratio per device class (iPhone 430:932, iPad 834:1194);
the frame scales to fit the stage height while preserving ratio.
- Shell radius 3.25rem (iPhone) / 2.75rem (iPad), screen radius
2.6rem / 2.1rem — the two-layer corner system.
- Percentage-based screen insets (iPhone 1.7% top / 2.5% sides /
1.9% bottom) matching Apple's actual bezel thickness.
- Dynamic island: 38% × 4.1%, positioned at 2.35% top, bg-base
rounded-full. Shows for iPhone only.
- Camera dot: 10×10, positioned at 1.45% top with a subtle ring.
Shows for iPad only.
- Inner bezel ring at inset 0.6% — catches light, hints at edge
thickness without adding chrome.
- iPhone vs iPad detection from pinned sim's name (same heuristic
apps/web uses).
We don't bundle the Apple-derived bezel assets (apps/web loads them
from a generated manifest via scripts/prepare-device-chrome.mjs —
needs Xcode + licensing considerations for packaged device-use);
the generic shell is the fallback Deus uses anyway when the manifest
isn't present, so visual parity is the same as Deus's default.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(device-use): drop overlay dynamic island/camera dot — stream already has them
The simulator's MJPEG stream captures iOS's full screen, including the
native status bar with its own dynamic island (iPhone) or camera
cutout (iPad). My overlay versions drew a second pill on top at
slightly different sizes/positions, creating the visible "two things
stacked" artifact the user flagged.
Fix: remove the overlay divs + their CSS. The stream shows the real
ones rendered by iOS at the right size/position.
Why Deus's apps/web draws them anyway: their code path assumes the
captured stream excludes the status bar (that is the case when they
eventually swap in the Apple-derived bezel assets, which crop it).
For the generic-shell + full-screen MJPEG case (what we have), the
overlays are redundant.
Kept: shell background, proper aspect ratio, shell/screen radii,
percentage screen insets, subtle inner ring. Those still make the
frame feel intentional without faking what's already in the stream.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(device-use): shell aspect ratio derived from stream so the screen fits exactly
The hardcoded 430/932 shell AR only matches the base iPhone 16 Pro.
For iPhone Air (420×912) or iPhone 17 Pro Max (440×956), the stream
either got stretched/squished by the canvas filling the box, or left
a black gap at the top/bottom because the inner screen box didn't
match the stream's aspect ratio.
Fix: compute the shell aspect ratio from streamInfo.size.ptW/ptH
so that, after subtracting the percentage bezel insets, the inner
screen box has exactly the same ratio as the stream. The canvas then
fills the screen cleanly with no object-fit letterboxing needed.
Math:
widthFrac = 1 − (left% + right%) / 100
heightFrac = 1 − (top% + bottom%) / 100
screenAR = ptW / ptH (from streamInfo.size)
shellAR = screenAR × widthFrac / heightFrac
Bezel insets (1.7/2.5/2.5/1.9 % for iPhone, 2.1 all-around for iPad)
are unchanged. Fallback to the previous hardcoded ratios when we
don't have streamInfo yet (empty-state placeholder). Dropped the
canvas `object-fit: contain` since there's nothing to contain now —
the box matches.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(device-use): collapsible sidebar with Panel wrapper
Extract ElementsPanel and ActivityPanel as discrete components under
components/sidebar/, share a Panel wrapper for chrome, and add a
collapsible shell with a Cmd+B toggle. Sidebar state persists via a new
ui-store (localStorage). Collapsed state shows a thin rail button on
the right edge.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: video-use + AAP v1.1 design draft
Working design doc covering (a) hyperframes investigation as a
candidate second AAP app and (b) AAP v1 protocol revisions
surfaced by that investigation. Not implemented yet —
informs upcoming PR sequence.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
zvadaadam
added a commit
that referenced
this pull request
May 31, 2026
17 inline comments from the bot — addressing all of them. Brief
mapping (file → comment id):
CRITICAL
package.json#62 Add missing zustand dep (was hoisted to root, would
fail bun install --frozen-lockfile in CI).
MAJOR
agentic-app.json#33 UI/MCP URLs use 127.0.0.1 instead of localhost so
host IDEs don't trip over IPv4/IPv6 happy-eyeballs
ambiguity. serve.ts also defaults --host to 0.0.0.0
(matches the server's own default), and the printed
probe URL collapses 0.0.0.0 → 127.0.0.1.
serve.ts#33 Compiled-layout candidate path was off-by-one
(..server vs ../..server). Fixed.
serve.ts#65 Browser only opens after /health responds 200; on
startup failure, no dead tab.
serve.ts#72 Propagate child exit code; non-zero exits return
{success:false} with the code.
e2e-server.ts#173 ChildProcess.killed is set when the signal is sent,
not when the process exits — using it as a kill-fallback
check would race a SIGTERM-ignoring child. Switched
to a tracked `exited` flag + clearTimeout on exit.
Sidebar.tsx#17 Refresh refs whenever streamUdid changes (sim swap),
not only when refs.length === 0. Stops stale refs
from a previous sim from lingering.
styles.css#85 Added --on-accent / --stage-bg / --frame-bg /
--canvas-bg / --shadow-* tokens; removed every
literal `white`, `#06060a`, `#1a1a22`, `#000`, and
`rgba(...)` from the stylesheet body.
state.ts#72 persist() now catches mkdir/writeFile failures
inside the chained promise so a single transient
error can't permanently brick the writeChain.
tools.ts#562 stream_logs wires onError + onExit to delete the
handle from logHandles, so failed/exited streams
don't leak.
MINOR
design-doc#57,#65 Architecture diagram said "TanStack Start on Nitro"
but the locked stack (decision #5) is "Hono on
Bun.serve" / "Vite + React". Aligned.
README.md#88 Added `text` language tag to the architecture fenced
code block (markdownlint MD040).
project-info.ts#43 detectKind() throws XcodebuildError on inputs that
aren't .xcodeproj or .xcworkspace, instead of
silently treating them as .xcodeproj.
xcodebuild.ts#159 Removed the synthesized appPath from BuildResult —
it was a guess based on `scheme.app` that's wrong
whenever target product name differs. Callers
should use resolveAppPath() (already used by the
`run` composite tool).
TopBar.tsx#15 onBoot pushes a failed tool-event into the activity
store on api.boot() failure, so the toast surfaces
the error.
sim-store.ts#67 setPinned checks res.success — only commits the
pinnedUdid change on success, otherwise stores
error state.
styles.css#345 Replaced deprecated `word-break: break-word` with
`overflow-wrap: anywhere` (modern equivalent).
Side effects:
scripts/*.ts Added `export {}` to the three smoke scripts so
tsc treats them as modules, not global scripts
(was causing duplicate-symbol errors after a
second smoke script joined the project).
Verified:
- 86/86 unit tests pass
- typecheck clean
- bun scripts/e2e-server.ts passes against a real iPhone 17 Pro
(17 tool-events, 97 tool-log frames, 0 failures)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
zvadaadam
added a commit
that referenced
this pull request
May 31, 2026
All 5 are doc-only fixes in docs/device-use-v2-design.md:
- Storage path duplication — `state.json` lives at
`{storage.workspace}/state.json`, not `{storage.workspace}/.device-use/state.json`
(the `.device-use` segment is already inside `storage.workspace`).
Fixed in 3 places (table row #12, backend wiring section).
- Architecture fence missing language — added `text` (markdownlint MD040).
- /ws bullet overstated CLI visibility — clarified that only actions
routed through the server (REST, MCP, viewer) populate the activity
stream; standalone CLI invocations bypass it (peer model).
- Phase 1 stack text was stale — said "Add TanStack Start" + "Nitro"
but the locked stack (decision #5) is Hono on Bun.serve + Vite.
Bullet rewritten to match what was actually built.
- Out-of-scope note about agentic-app.json was outdated — the manifest
IS in this PR; the next PR is the Deus host that consumes it. Reworded
+ retitled the manifest section to match.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
zvadaadam
added a commit
that referenced
this pull request
May 31, 2026
…olor), un-exported 3 internal-only symbols (RECENT_PROJECT_LIMIT, resolveGitProjectRoot, setLastOpenInAppId), and replaced duplicate timeAgo in AccessSection.tsx with shared formatTimeAgo — net reduction of 41 lines across 5 files with all 825 tests passing.
zvadaadam
added a commit
that referenced
this pull request
May 31, 2026
…safe parser in shared/messages/, added parts field to the shared Message type, and aligned backend persistence serialization with the shared schema — 14 new tests, all 459 backend + 371 agent-server tests passing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
When the console panel is opened in the browser, the layout breaks, causing elements to shift incorrectly and become inaccessible. This is due to improper flexbox constraints on the console panel itself, which affects the layout of the main browser view and toolbar.
Changes
flex-shrink-0to the main console panel container insrc/features/browser/components/BrowserPanel.tsx. This ensures the console panel maintains its designated height and does not shrink or expand in a way that disrupts the rest of the layout.min-h-0on the main browser view to ensure it properly takes up available space without being pushed by the console.Testing
Greptile Overview
Updated On: 2025-10-18 11:49:17 UTC
Greptile Summary
This PR introduces a comprehensive browser integration feature to the Box IDE while addressing layout stability issues when the console panel is opened. The core change implements a complete embedded browser system with cross-origin handling, MCP server integration for AI-powered browser automation, and a collapsible console panel. The browser functionality includes navigation controls, iframe sandboxing, automation script injection, and DevTools-style console logging.
The implementation spans multiple architectural layers: a new BrowserPanel React component, Tauri-based BrowserManager for process management, backend API extensions for health checks and port discovery, and supporting hooks for dual-mode operation (desktop/web). The PR also restructures the Dashboard's right panel into a tabbed interface to accommodate the new browser tab alongside existing Changes and Terminal tabs.
Key infrastructure changes include dynamic port discovery for robust backend connections, removal of hardcoded UI spacing constraints that were causing layout conflicts, and text overflow improvements in chat messages. The browser automation follows established patterns from Cursor IDE, using HTTP MCP servers for AI control while maintaining security through iframe sandboxing.
Important Files Changed
Changed Files
src/features/browser/components/BrowserPanel.tsxsrc/Dashboard.tsxsrc-tauri/src/browser.rssrc-tauri/src/main.rssrc-tauri/src/commands.rssrc/features/browser/hooks/useDevBrowser.tssrc/config/api.config.tssrc/components/ui/sidebar.tsxsrc/App.csstsconfig.jsonbackend/server.cjs/api/healthroutes.mcp.jsonsrc/components/ui/tabs.tsxsrc-tauri/src/lib.rssrc-tauri/src/socket.rssrc/features/workspace/components/MessageItem.tsxsrc/features/browser/components/index.tscursor-final.mdBROWSER_INTEGRATION.mdcursor-web-mcp-connect-analysis.mdcursor-web-analysis.mdBROWSER_TEST_RESULTS.mdtest-page.htmlConfidence score: 3/5
.mcp.json, conflicting API routes inbackend/server.cjs, hardcoded development paths in browser hooks, and relaxed TypeScript linting rules.mcp.jsonsyntax errors, backend route conflicts, and the hardcoded paths inuseDevBrowser.tsSequence Diagram
sequenceDiagram participant User participant BrowserPanel participant ConsolePanel participant IframeContainer participant DevBrowser Note over User,DevBrowser: Browser Panel Layout with Console - Fix Flow User->>BrowserPanel: Click Terminal icon to toggle console BrowserPanel->>BrowserPanel: setShowConsole(!showConsole) alt Console Opening BrowserPanel->>ConsolePanel: Render console panel with flex-shrink-0 Note over ConsolePanel: Fixed height container prevents layout shift ConsolePanel->>ConsolePanel: Apply min-h-[100px] max-h-40 constraints BrowserPanel->>IframeContainer: Adjust flex-1 min-h-0 for remaining space Note over IframeContainer: Takes available space without being pushed else Console Closing BrowserPanel->>ConsolePanel: Remove console from DOM BrowserPanel->>IframeContainer: Reclaim full flex-1 space Note over IframeContainer: Expands to full browser view height end User->>BrowserPanel: Navigate to URL while console open BrowserPanel->>DevBrowser: Start MCP server if needed BrowserPanel->>IframeContainer: Set iframe src Note over IframeContainer: Layout remains stable during navigation IframeContainer->>BrowserPanel: onLoad event BrowserPanel->>BrowserPanel: Auto-inject automation script BrowserPanel->>ConsolePanel: Add log messages ConsolePanel->>ConsolePanel: Auto-scroll to bottom Note over ConsolePanel: Console panel maintains fixed constraints User->>BrowserPanel: Test with sidebar collapsed/expanded BrowserPanel->>BrowserPanel: Maintain flex layout integrity Note over BrowserPanel: flex-shrink-0 on console prevents compression Note over IframeContainer: min-h-0 ensures proper space calculation User->>User: Verify toolbar, status bar, and browser view accessible Note over BrowserPanel: Layout fix ensures all elements remain positioned correctlyContext used:
dashboard- CLAUDE.md (source)Summary by CodeRabbit
New Features
Documentation
Chores / Tests
Style