Skip to content

feat(desktop): interactive pet action center overlay - #70156

Closed
s-a-s-k-i-a wants to merge 18 commits into
NousResearch:mainfrom
s-a-s-k-i-a:feat/interactive-pet-action-center
Closed

feat(desktop): interactive pet action center overlay#70156
s-a-s-k-i-a wants to merge 18 commits into
NousResearch:mainfrom
s-a-s-k-i-a:feat/interactive-pet-action-center

Conversation

@s-a-s-k-i-a

Copy link
Copy Markdown

Adds a gateway-less action center to the pet overlay window — pending approvals, clarify requests, and live-turn items surface in the popped-out pet without pulling focus or auto-opening. The panel lays out dynamically (above/below/left/right + start/center/end alignment) based on the pet's screen position and available work-area space, keeping the visible sprite anchor stationary through every open/close cycle.

Key pieces:

  • computeActionCenterPlacement() + computeActionCenterAlignment() — measure actual panel overflow against work area, not a fixed edge threshold
  • petOverlayTargetOffset() — two-layer offset: Electron window positioning + React content offset, computed independently
  • petAnchorRef tracks the confirmed sprite anchor; drag-while-open preserves it through resize
  • Height parity added to overlayWindowTargetSize() to prevent half-pixel drift on both axes

Tests: 69 focused (13 geometry + 34 store + 22 app-integration), full suite 1513 UI + 426 Electron, typecheck/lint/build clean. E2E CDP smoke verified 0px sprite drift.

Closes s-a-s-k-i-a#2

Projection slice (pet-action-center.ts):
- Global approval/clarify projection into PetActionCenterState
- Sudo/secret contribute only aggregate secureInputCount/
  attentionCount/blockingCount — no per-request item/id/profile/
  session/prompt/requestId is serialized
- Deterministic itemId, stable sort, validated selection fallback
- Pure JSON-serializable state, no prose or raw backend errors

IPC parser (pet-overlay.ts):
- parsePetOverlayControl() narrows untrusted overlay payloads into
  typed discriminated controls (approval/clarify/select/open)
- Strips injected profile/session/route fields; malformed payloads
  ignored; onControl callback typed as unknown in global.d.ts

Handler dispatch (pet-action-center-actions.ts):
- Main-renderer action mutex prevents duplicate side-effect RPCs
- Approval: origin gateway, current-item capability validation,
  denial reason only for deny, strict response decoder
  (resolved:true → success, false/0 → stale, other → rpc-failed)
- Clarify: original backend requestId, strict response decoder
  (status:'ok' → success, exact stale error → stale, other →
  rpc-failed); empty answer for clarify-skip
- Replacement generation survives; exact generation cleared only
  on confirmed success
- Exact open: profile+storedSessionId through resumeSession;
  reverify item/capability/session after awaited ensureProfile

Resolver (utils.ts):
- resolveStoredSession(id, expectedProfile) filters cache and REST
  to the expected profile; profile-aware upsert preserves same
  stored ID in other profiles
- resumeSession optional third expectedProfile argument; returns
  Promise<boolean> — true only on real warm/gateway resume

78 focused tests across 7 files; 56 adjacent route/prompt tests;
typecheck renderer+electron; eslint; production build green.

Spec-compliance: 3 independent reviews PASS/APPROVED.
Project only assistant text from pet-started turns into the secondary overlay, with profile/session-scoped arming, terminal-event race handling, bounded output, and accessible rendering.
Keep pop-out geometry anchored across action-center resizing, clustered zoom gestures, dragging, and display reclamps. Route exact-session opens through the canonical navigation owner and expand regression coverage for stale native bounds responses.
Close the overlay action center as soon as Open in Hermes routes to the main window. This prevents asynchronous route and live-status updates from producing multiple native resize transitions.
The pet overlay window now dynamically chooses its expansion direction
based on available screen space, like the Codex pet:

- Pet at top edge → action center expands downward (top edge stays)
- Pet at bottom edge → expands upward (bottom edge stays, current behavior)
- Pet at left edge → expands rightward (left edge stays)
- Pet at right edge → expands leftward (right edge stays)
- Pet centered → symmetric expansion (current behavior)

Two changes make this work:

1. Edge-tolerant clamping (pet-overlay-geometry.ts): clampPetOverlayBounds
   now accepts spriteSafeMarginX/Y options that let the transparent window
   overlap the work-area edge by the sprite padding amount, so the sprite
   can reach x=0/y=0 instead of being held back by padding margins.

2. Expansion direction (pet-overlay.ts): new computeExpansionDirection()
   picks up/down/left/right/center from the pet's current screen position
   vs the work area. anchoredOverlayBounds() uses it to anchor the
   appropriate edge instead of always bottom-center.

Closes #2
Anchor native bounds to the rendered sprite group so opening and closing the action center preserves the visible pet rectangle at every screen edge. Flip the renderer layout inward, account for compact bubble offsets, and prevent stale expansion responses from releasing the anchor during collapse.

Verified with 1507 UI tests, 426 Electron tests (1 skipped), lint, typecheck, production build, and real Electron Open→Close cycles at top and left edges.
@s-a-s-k-i-a

Copy link
Copy Markdown
Author

Superseded by the issue-first, maintainer-compatible rebuild. The first independently reviewable replacement is #70226 (approval denial-reason propagation). The remaining behavior is tracked in s-a-s-k-i-a#4 with child slices s-a-s-k-i-a#7, s-a-s-k-i-a#8, and s-a-s-k-i-a#9, followed by s-a-s-k-i-a#5, s-a-s-k-i-a#6, and s-a-s-k-i-a#2.\n\nClosing this broad, conflict-heavy PR intentionally so reviewers see only current-main, test-first slices. The original branch remains preserved for behavior reference; code is being ported semantically rather than merged wholesale.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/desktop Electron desktop app (apps/desktop/*) comp/tui Terminal UI (ui-tui/ + tui_gateway/) needs-repro Bug needs reproduction steps P3 Low — cosmetic, nice to have sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Umbrella] Adaptive Pet-Geometrie, native Bounds und Real-Electron-Verifikation

2 participants