feat(app-control): wire "redo" op into the BACKGROUND action (#10694) - #10960
Merged
Merged
Conversation
The UI redo half shipped in PR #10771 (useBackgroundApplyChannel already handles op:"redo" via redoBackgroundConfig), but the agent-side emitter silently dropped "redo the background" — there was no redo path in inferBackgroundPlan, so chat could undo but never redo. Mirror the existing undo op end to end: - add "redo" to BackgroundApplyOp / the BackgroundPlan union and the op parameter enum so it matches the UI contract; - add REDO_RE and detect it BEFORE color resolution (so "red" inside "redo" can't false-match a color), alongside undo/reset; - map the redo plan to emit { op: "redo" }; - add REDO_BACKGROUND simile + a redo example. Real tests: "redo the background" / "go forward on the background" infer op:"redo" and emit { op:"redo" }; "red" still resolves to a color set, not redo. Fail-without-fix verified (removing the detection reds the 3 new tests, greens the rest). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
lalalune
pushed a commit
that referenced
this pull request
Jul 1, 2026
Resolve 3 conflicts: - plugins/plugin-app-control/src/actions/background.ts + .test.ts: take develop's #10960 BACKGROUND redo action (more thorough: re-apply|go forward + RESET_RE exclusion). PR's unique settings-UI redo button in BackgroundSettingsControls.tsx (non-conflicting) preserved. - packages/ui/src/components/shell/__e2e__/run-chat-sheet-e2e.mjs: take develop's #10972 real-touch comment; gesture code identical on both sides and already drives touchDragHold real-touch path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
lalalune
pushed a commit
that referenced
this pull request
Jul 2, 2026
…10694) develop already shipped the versioned undo/redo half of #10694 (#10771 + #10960). This adds the headline feature it was missing: a real, arbitrary GLSL fragment-shader background mode. Renderer (@elizaos/ui/backgrounds): - shader-schema.ts — typed, clamped uniform contract (u_speed/scale/intensity/ seed), hexToRgb, and a static fragment-source safety gate (size bound + reject unbounded while/do loops). three.js-free so it unit-tests without WebGL. - shader-presets.ts — 5 self-contained GLSL ES 1.00 presets (aurora, lava, plasma, waves, nebula), bounded loops only. - ProgrammableShaderBackground.tsx — renders the fragment via raw three.js (already a dep; no new deps) with the full safety sandbox required for untrusted GPU code: compile-validate BEFORE showing, uniform clamping, a frame-time watchdog that falls back on a GPU stall, WebGL context-loss recovery, and a single static frame under prefers-reduced-motion. Any failure → onFallback and the color field paints instead — a bad shader can never white-screen or hang the app. - BackgroundConfig gains a `glsl` mode (source + presetId + uniforms); normalizeBackgroundConfig validates/clamps it and collapses a missing/hostile/oversized source to the color field on load. - AppBackground switches to the shader (keyed by source for a fresh attempt on replacement) with the color-field fallback; useBackgroundApplyChannel handles glsl payloads: preset set, explicit source, and live-shader uniform tweaks. Agent (plugin-app-control BACKGROUND action): - NL → shader plan: named presets (+ synonyms: molten→lava, ocean→waves, cosmic→nebula), a generic "animated shader" → default preset, and relative tweaks (slower/faster/brighter/dimmer/bigger/more-detailed) → uniform patch. Precedence: named preset > concrete color > tweak > generic shader, so "red brighter" stays red and "molten" beats a bare color. The action names a preset id + uniforms only; GLSL source stays in the renderer. Tests (all green): shader-schema (clamp/NaN/hostile/while-loop/hex), preset library integrity, ProgrammableShaderBackground fallback + no-crash-on-garbage, persistence glsl round-trip + hostile-source collapse, AppBackground glsl render + fallback, a 400-step set/undo/redo storm fuzz across shader/image/glsl asserting clamped-finite uniforms + history-cap + flag invariants, and BACKGROUND action preset/tweak inference + emit. Stories added for story-gate. ui + plugin-app-control typecheck + biome clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
lalalune
pushed a commit
that referenced
this pull request
Jul 2, 2026
…#10694) The rebase onto develop merged both develop's original standalone redo button (#10960) and this branch's paired undo+redo layout into BackgroundSettingsControls — two "Redo background change" buttons sharing the same background-redo agent-element ref. Kept the paired-disabled layout (undo + redo shown together, redo disabled without redo history) and removed the superseded standalone. BackgroundSettingsControls.test.tsx 4/4 green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
lalalune
added a commit
that referenced
this pull request
Jul 2, 2026
…ckground + interaction/QA de-larp (#9952, #10694, #10722, #10434) (#11083) * feat(app-control/ui): give background redo its real triggers — agent op + settings button (#10694) The #10771 slice landed redo in the store and the background:apply receiver, but nothing could emit it: the BACKGROUND action op union stopped at set|undo|reset and the settings controls only rendered Undo. - BACKGROUND action: add "redo" to the op union/param enum, REDO plan inference (checked before UNDO so "restore the change I undid" reads as redo), handler branch broadcasting {op:"redo"}, REDO_BACKGROUND simile, examples + docs. - Settings: Redo2 button paired with Undo, disabled without redo history, registered as agent-surface element background-redo. - Tests: 5 new action tests (21/21 green; plugin suite 1262 green), new BackgroundSettingsControls.test.tsx, BackgroundView redo case, fuzz/store seeds carry the redo fields. - background e2e: fixture mirrors useDisplayPreferences redo semantics; runner gains set→set→undo→redo round-trip (agent redo + UI redo) with refreshed screenshots/video evidence (11/11 assertions green). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ui): de-larp gesture e2e to real CDP touch — and fix the two real bugs it exposed (#10722) CI gated the synthetic-event gesture runner while the real-touch runner was excluded — the gated test was the larp. Converting to real input immediately surfaced two real production bugs the synthetic events masked: - TopicGroup: pointerup→onTap toggles, React swaps header↔pill, and the browser's synthesized click re-hit-tests onto the replacement element and toggles BACK — a tap/click on the header/pill was a visible no-op on phones and desktop. Fixed with the suppressClickRef + onClickCapture idiom (onClick kept for keyboard activation). - HomeScreen: the home scroller (overflow-y-auto ⇒ touch-action:auto) consumed horizontal touch pans itself, so home→launcher swipe could never fire from the widget area on real touch. Added touch-pan-y. Test/CI changes: - run-chatux-gesture-e2e: synthetic PointerEvent dispatch → shared touchTap/touchSwipe (real CDP touch) + real mouse flicks for the fine-pointer profile; assertions now match real behavior (vertical touch flick on a pan-y header scrolls, it does not collapse). - run-home-screen-e2e: the 3 mobile outer-rail swipes converted to real touch (+touchSwipeRight); desktop rail paging deliberately stays mouse (fine-pointer affordance, documented). - real-touch-gestures: settleMainThread (rAF round-trip) before every gesture — root-causes and fixes a pre-existing develop red where CDP touch raced a busy renderer main thread in conversation-swipe; deleted the zero-consumer touchPinch export (touchTap now consumed). - ui-e2e-gate.yml: gate test:home-screen-e2e (the honest runner is now in CI), fix the mislabeled chatux step, add output-home artifacts + src/testing path filters. Verified locally: chatux 7/7, home-screen full pass (~70 asserts), conversation-swipe ALL PASSED (5 failed on develop baseline before the helper fix), chat-sheet pass, 40/40 unit tests, Biome clean, workflow YAML parsed. Evidence re-captured in output-home/ and output/. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(agent/app-core): de-larp #10712 — honest SSE contract + live streaming/thinking assertions with real Cerebras evidence The "local-inference vs cloud-resolved provider path" parity matrix was a labeled mock: the providerPath string only round-tripped through the test's own fixtures, and no code under test reads a provider branch (route → generateChatResponse → messageService → useModel is provider-agnostic; selection lives in core's model registry, which the contract test mocks by design). Collapsed to one honestly-named conversation-stream-sse-contract.test.ts pinning the full frame contract: thinking→streaming status ordering, ordered cumulative tokens, terminal done with fullText+thought, done-is-terminal, thought-never-in-visible-text. The real provider path is now covered where it belongs — the live e2e: - streaming-visible-text.live.e2e.test.ts records every SSE payload and adds a live frame-contract test (status ordering, monotonic fullText, done schema, conditional-but-contractual thought that must never leak into visible frames). - Fixed a real warm-boot race it exposed: /api/health flips ready before the deferred provider plugin registers — gate on /api/status.canRespond instead. - Hardened the lane: persistent models cache, 300s ready timeout, hang-proof teardown, ELIZA_STREAM_FRAME_DUMP evidence hook. Live evidence (hand-read, key-free logs) in .github/issue-evidence/10712-live-streaming-thinking/: 3 consecutive green runs against Cerebras gpt-oss-120b, 19 token frames growing 43→909 chars, real usage (2737 prompt / 229-234 completion tokens). Honest findings recorded: live reply arrives via running_action (REPLY callback), and thought was absent on this path in both dumped runs — the thought-carrying branch stays pinned deterministically. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(app): give the Her-minimal aesthetic gate teeth — ratchet with committed baseline (#9950) The minimalism metrics were computed but could never block: computeVerdict mapped every breach to needs-eyeball, and the signal was actually DEAD in real runs — the audit spec never passed viewportArea, so exceedsMinimalismBudget always returned false. A cramped view shipped green. Ratchet design (same idiom as ui-determinism-baseline): - aesthetic-minimalism-baseline.json records breaching views' metrics (83 entries / 33 slugs from a real full audit:app run on this tree). - Breaching view NOT in baseline → needs-work (BLOCKING). - Baselined view → blocks only when a metric regresses past baseline ±5% relative tolerance (live-DOM jitter absorber; real regressions move density far past 5% — recorded values cluster at 89-304 vs the 45 ceiling). Within tolerance stays soft. - Spec afterAll now throws unconditionally on ratchet failures (verified live: the empty-baseline run correctly failed on all 83). Corrupt baseline fails loudly via parseMinimalismBaseline. - Refresh: bun run --cwd packages/app audit:app:minimalism:update (regenerates from the latest report.json; prunes healed views so a re-breach counts as new). Also: collectHoverViolations no longer swallows hover failures with .catch(()=>{}) — failed probes are recorded as hoverFailures findings. First real run surfaced 8 previously-invisible probe failures. Verified: full audit:app run (348 passed, gate replay against committed baseline → ratchet-blocking=0, good=136/needs-eyeball=212); 55/55 audit unit tests incl. 12 new ratchet cases; Biome clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(ui): lock the floating chat to onboarding until an option is chosen, auto-collapse on completion (#9952 follow-up) During in-chat first-run the overlay was fully interactive: free text went to the real server before a runtime existed, and Escape/outside tap/pull-down could dismiss the onboarding sheet entirely (the e2e helpers relied on it). Now, while first-run is pending: - Composer locked: textarea disabled with "Choose an option to continue"; attach/mic/send (and push-to-talk) disabled; submitText hard-guarded against the prefill/dictation/slash side doors. The transcript choice widgets and OAuth/secret blocks stay interactive. - Send seam hard-blocked: sendActionMessage drops any non-__first_run__: value while firstRunComplete is false — nothing reaches rawSendActionMessage. - Collapse locked at one choke point plus drag gates: document/thread/ composer Escape, grabber tap + keyboard close, outside-tap detector, pull-down/drag/settle-free, header clear + launcher buttons, and conversation swipe are all no-ops; the sheet stays pinned at full. - Auto-collapse exactly once on the completion falling edge (wasFirstRunOpenRef), revealing home; ordinary sessions never collapse. Desktop ?shellMode=chat-overlay verified unaffected — that shell never mounts the overlay/conductor. - Dead code: finishRemote + normalizeRemoteTarget deleted; runFirstRunFinish narrowed to FirstRunFinishDraft (cloud|local); dead switchAgentProfile port unwired (live remote path is adopt-remote-first-run, untouched). Tests: new ContinuousChatOverlay.firstrun.test.tsx (8 — lock, no-collapse paths, choice interactivity, one-shot auto-collapse + unlock); conductor state-machine tests now drive the real handleFirstRunAction seam (local end-to-end + POST-once + re-tap idempotency, BYO-keys banner, cloud OAuth→agent-pick→bind→tutorial, cloud error→local retry, unmount unregistration); e2e asserts the lock + negative Escape mid- onboarding and auto-collapse + unlock in all five completion flows (14 passed incl. Pixel-7 touch lane); cloud-live spec picks tutorial:skip before its real turn since completion now gates the composer. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(agent,ui): remove the dead custom-VRM / custom-background upload routes (#10434 follow-up) The 3D companion (VRM avatar + its scene background) was removed in #10434, but the plumbing that fed it was left behind: the POST/GET `/api/avatar/vrm` + `/api/avatar/background` routes, the client's uploadCustomVrm/hasCustomVrm/uploadCustomBackground/hasCustomBackground methods (upload* had no callers at all), and the startup existence probes. Nothing renders an uploaded model or scene background anymore, so those HEAD probes only produced 404 noise on every first-run/startup. - avatar-routes.ts: drop the four vrm/background handlers (keep the live Discord avatar cache route); drop the now-unused readRequestBodyBuffer + resolveStateDir imports and the unused req/json ctx destructure. - client-agent.ts: delete the four dead ElizaClient methods + their type declarations. - startup-phase-hydrate.ts: drop the hasCustomVrm/hasCustomBackground probes (and the resolveApiUrl import + two Date.now() render-time nondeterminism sources they carried), keeping the live `resolvedIdx === 0 -> setSelectedVrmIndex(1)` default-avatar fallback. Live built-in avatar selection (selectedVrmIndex / avatarIndex / getVrmPreviewUrl) and the content-pack asset path are untouched. Agent + UI typecheck clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ui,app-control): arbitrary programmable GLSL shader background (#10694) develop already shipped the versioned undo/redo half of #10694 (#10771 + #10960). This adds the headline feature it was missing: a real, arbitrary GLSL fragment-shader background mode. Renderer (@elizaos/ui/backgrounds): - shader-schema.ts — typed, clamped uniform contract (u_speed/scale/intensity/ seed), hexToRgb, and a static fragment-source safety gate (size bound + reject unbounded while/do loops). three.js-free so it unit-tests without WebGL. - shader-presets.ts — 5 self-contained GLSL ES 1.00 presets (aurora, lava, plasma, waves, nebula), bounded loops only. - ProgrammableShaderBackground.tsx — renders the fragment via raw three.js (already a dep; no new deps) with the full safety sandbox required for untrusted GPU code: compile-validate BEFORE showing, uniform clamping, a frame-time watchdog that falls back on a GPU stall, WebGL context-loss recovery, and a single static frame under prefers-reduced-motion. Any failure → onFallback and the color field paints instead — a bad shader can never white-screen or hang the app. - BackgroundConfig gains a `glsl` mode (source + presetId + uniforms); normalizeBackgroundConfig validates/clamps it and collapses a missing/hostile/oversized source to the color field on load. - AppBackground switches to the shader (keyed by source for a fresh attempt on replacement) with the color-field fallback; useBackgroundApplyChannel handles glsl payloads: preset set, explicit source, and live-shader uniform tweaks. Agent (plugin-app-control BACKGROUND action): - NL → shader plan: named presets (+ synonyms: molten→lava, ocean→waves, cosmic→nebula), a generic "animated shader" → default preset, and relative tweaks (slower/faster/brighter/dimmer/bigger/more-detailed) → uniform patch. Precedence: named preset > concrete color > tweak > generic shader, so "red brighter" stays red and "molten" beats a bare color. The action names a preset id + uniforms only; GLSL source stays in the renderer. Tests (all green): shader-schema (clamp/NaN/hostile/while-loop/hex), preset library integrity, ProgrammableShaderBackground fallback + no-crash-on-garbage, persistence glsl round-trip + hostile-source collapse, AppBackground glsl render + fallback, a 400-step set/undo/redo storm fuzz across shader/image/glsl asserting clamped-finite uniforms + history-cap + flag invariants, and BACKGROUND action preset/tweak inference + emit. Stories added for story-gate. ui + plugin-app-control typecheck + biome clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): gate the last onboarding-lock collapse seam + document the contract (#9952) Cluster-A audit follow-up. Two gaps on the onboarding lock: - The TUTORIAL_CHAT_CONTROL_EVENT handler (rest/reset → collapse, prefill → un-pill) was the one collapse seam outside the gated funnel. It is unreachable in the real flow (the tour starts only after completeFirstRun), but a stray or adversarial window event during onboarding could have collapsed the pinned sheet. Now the handler no-ops while firstRunOpen. New firstrun test drives rest/reset/pill/prefill during onboarding and asserts the sheet stays pinned FULL and the composer stays locked. - The lock contract had zero documentation. Documented the full contract (pinned FULL, composer + side-door lock, every gated collapse path, one-shot auto-collapse) in first-run/IN_CHAT_ONBOARDING_DESIGN.md + a README pointer. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui,app): strict-typecheck cleanups + de-larp the aesthetic density probe (#9950, #10712) - #9950: the all-views aesthetic audit caught a crashed density probe with a zero-density placeholder ({borderDividerCount:0, whitespaceRatio:1}), which SILENTLY satisfied both the minimalism budget and the ratchet — a probe crash read as "perfectly minimal → pass". Now the failure is recorded as a densityProbeFailures finding (surfaced in the summary log + per-view markdown, same posture as hoverFailures) and the budget/ratchet are skipped when the probe failed, so a crash can never manufacture a pass. - #10712: force-add the four gitignored live-run streaming logs the evidence README cites (git add -f) so a fresh clone keeps the real-Cerebras run proof. - Strict-config typecheck fixes for the #10694/#10434 work: widen makeGlslConfig's uniforms param to the unknown-valued partial it already normalizes, and drop the two now-removed avatar setters (setCustomVrmUrl/ setCustomBackgroundUrl) from the useStartupCoordinator deps literal in AppContext (they left HydratingDeps with the dead-route removal). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(spatial): de-larp the WEBXR "validated end-to-end" overclaim (#10722) Cluster-E audit: WEBXR_PLATFORMS.md claimed the production enterImmersiveScene() was "validated end-to-end against the IWER emulator … read back two textured quads from the session framebuffer", but no committed test opens an immersive session or reads pixels — the only test exercises the mocked navigator.xr availability contract — and enterImmersiveScene/enterImmersiveFromSpecs have no production caller. Removed the unbacked claim and added an explicit "NOT yet validated" section stating the immersive render path + hand-pose interaction are uncovered and the entry points are unused, so the doc no longer advertises a validation that doesn't exist. (The sibling tui-pty "vacuous skip" audit finding was a false negative: the real-PTY suite IS wired into CI via test.yml's "Real-PTY TUI smoke" step — `test:tui-pty` sets RUN_TUI_PTY=1 internally — and passes locally; left as-is.) Also bundles the four force-added #10712 live-run streaming logs the evidence README cites (were gitignored). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): drop the duplicate background redo button left by the rebase (#10694) The rebase onto develop merged both develop's original standalone redo button (#10960) and this branch's paired undo+redo layout into BackgroundSettingsControls — two "Redo background change" buttons sharing the same background-redo agent-element ref. Kept the paired-disabled layout (undo + redo shown together, redo disabled without redo history) and removed the superseded standalone. BackgroundSettingsControls.test.tsx 4/4 green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Shaw <shawgotbags@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Closes a dead chat contract in #10694. The versioned undo/redo UI shipped in PR #10771 —
useBackgroundApplyChannelalready handlesop === "redo"→redoBackgroundConfig(), andBackgroundApplyOpon the UI side is"set"|"undo"|"redo"|"reset". But the agent-sideBACKGROUNDaction (plugins/plugin-app-control/src/actions/background.ts) had no redo path — its op union, plan, regex, handler, and examples all stopped atundo/reset. So "redo the background" from chat inferred no plan and silently broadcast nothing; the background stayed stuck on the undone state.Fix (mirror of the existing
undoop)"redo"added toBackgroundApplyOp+BackgroundApplyPayload(matching the UI union) and a{ op: "redo" }BackgroundPlanvariant.REDO_RE = /\b(redo|re-?do|go forward|step forward|re-?apply)\b/i, detected ininferBackgroundPlanbefore color resolution and guarded with!RESET_RE(exactly likeUNDO_RE) — so"red"inside"redo"can never false-match a color.{ op: "redo" }→ the already-merged channel callsredoBackgroundConfig(); reply "Re-applied the background you undid."REDO_BACKGROUNDsimile, a redo example, and updated description/enum.Tests
background.test.ts+6:"redo the background"/"go forward"inferop:"redo"and broadcast{op:"redo"}; a color word ("red") still resolves to a color set, not redo.Fail-without-fix proven (independently re-verified): reverting only
background.tsreds the 3 new redo tests (the"red"→color test still passes, confirming no false-match); restoring greens all 21.Evidence
On-device (Android 14 emulator) — the chat flow blue → green → undo (→blue) → "redo the background" stepping forward to green, over the redone background, with the before-note ("stuck on blue") —
.github/issue-evidence/10694-background-redo-android.png. Real render surface: the redo path drives the always-mountedAppBackgroundvia the shipped channel.Scope
This is the redo-op emitter slice — the clean, self-contained half #10771 left open. The headline programmable GLSL shader mode (real WebGL + uniform schema + preset library + GPU safety) is a separate multi-file epic, explicitly out of scope here.
🤖 Generated with Claude Code