Skip to content

feat(a2ui): use pexels to generate images#2684

Merged
Sherry-hue merged 1 commit into
lynx-family:mainfrom
Sherry-hue:feat/genui-server-image
May 21, 2026
Merged

feat(a2ui): use pexels to generate images#2684
Sherry-hue merged 1 commit into
lynx-family:mainfrom
Sherry-hue:feat/genui-server-image

Conversation

@Sherry-hue
Copy link
Copy Markdown
Collaborator

@Sherry-hue Sherry-hue commented May 21, 2026

Summary by CodeRabbit

  • New Features

    • Generated messages now include resolved image URLs (Pexels when available; deterministic fallback otherwise).
  • New Behavior

    • Chat streaming suppresses intermediate preview messages while still streaming text.
  • Updates

    • Prompt wording refined to include “photo” for weather and product suggestions; image queries are normalized for better matches.
  • Documentation

    • Added guidance on image resolution and Pexels API configuration.

Review Change Stack

Checklist

  • Tests updated (or not required).
  • Documentation updated (or not required).
  • Changeset added, and when a BREAKING CHANGE occurs, it needs to be clearly marked (or not required).

@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented May 21, 2026

⚠️ No Changeset found

Latest commit: abda536

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 21, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 0a677371-d693-47a0-a776-be3f0f52503a

📥 Commits

Reviewing files that changed from the base of the PR and between 03c5e0c and abda536.

📒 Files selected for processing (7)
  • packages/genui/a2ui-playground/src/pages/AIChatPage.tsx
  • packages/genui/server/AGENTS.md
  • packages/genui/server/agent/a2ui-prompt.ts
  • packages/genui/server/agent/image-resolver.ts
  • packages/genui/server/app/a2ui/action/stream/route.ts
  • packages/genui/server/app/a2ui/stream/route.ts
  • packages/genui/server/service/a2ui-agent.ts

📝 Walkthrough

Walkthrough

Server resolves A2UI Image component queries to concrete URLs (Pexels with deterministic Picsum fallback) and integrates resolution into validation/stream completion paths. Client-side readA2UIResponse gains a publishPartialMessages option to defer intermediate message-array publishing during SSE streaming; suggested prompts updated.

Changes

Image URL Resolution and Streaming Integration

Layer / File(s) Summary
Image resolution core implementation
packages/genui/server/agent/image-resolver.ts, packages/genui/server/agent/a2ui-prompt.ts, packages/genui/server/AGENTS.md
Adds resolveA2UIImageUrls with two‑phase pointer resolution, query normalization, LRU cache of promise results, Pexels lookup with abortable request, deterministic picsum.photos fallback, JSON‑pointer helpers, and recursive image-like field resolution. Updates system prompt rule and docs about PEXELS_API_KEY and fallback behavior.
Server route and service integration
packages/genui/server/app/a2ui/action/stream/route.ts, packages/genui/server/app/a2ui/stream/route.ts, packages/genui/server/service/a2ui-agent.ts
Wires the resolver into post-validation success paths: when validateA2UIOutput returns ok, validation.messages is replaced with await resolveA2UIImageUrls(...) before emitting or returning; on validation failure, messages become an empty array.
Client-side streaming control
packages/genui/a2ui-playground/src/pages/AIChatPage.tsx
readA2UIResponse accepts options.publishPartialMessages (default true); when set false, SSE delta handling skips emitting intermediate parsed message arrays while still streaming text via onText. handleSend calls the reader with { publishPartialMessages: false }. Suggested prompt copy updated to mention "a photo".

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

Suggested reviewers

  • PupilTong
  • gaoachao
  • HuJean

Poem

🐰 I tidy queries, sniff and hop,
To Pexels I leap — if not, Picsum on top.
URLs land tidy, cached in a row,
Streams hold their breath while the final ones flow.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main feature added: integration with Pexels API for image generation/resolution in A2UI.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/genui/a2ui-playground/src/pages/AIChatPage.tsx`:
- Line 433: The quiz prompt string in AIChatPage.tsx contains a grammatical
error ("and photos" inserted into the answer list); update the prompt used where
the string is defined so the answer list reads cleanly (e.g., "Triangle, Square,
Circle, Hexagon") or, if you intended photos for an option, clarify like "Circle
(with photos)" so "and photos" is not breaking the list; locate and edit the
prompt text used in the Create a trivia quiz card string.

In `@packages/genui/server/agent/image-resolver.ts`:
- Line 26: The global imageCache (const imageCache = new Map<string,
Promise<string>>()) is unbounded and must be limited; replace it with a bounded
LRU-style cache (or use a small dependency like lru-cache) and apply the same
change to the other cache/location referenced around lines 107-113.
Specifically, create a capped cache that evicts oldest entries when max size is
reached (e.g., max entries 1000 or configurable), keep the same key/value type
(string -> Promise<string>), and update any code that accesses imageCache to use
the new cache API (get/set/has or .get/.set for lru-cache); ensure race
conditions are preserved by storing the Promise as before.
- Around line 126-128: The Pexels request (the line with "const res = await
fetch(url, { headers: { Authorization: apiKey } })") needs an AbortController
timeout so a hung upstream doesn't leave the cached Promise unresolved; create
an AbortController, start a setTimeout to call controller.abort() after a
sensible timeout, pass controller.signal into fetch, and in a finally block
clear the timeout; catch aborts/errors and return null on failure so the
existing picsum fallback runs.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 40167e12-0bc6-42fa-b0e5-a8ac90aa2d44

📥 Commits

Reviewing files that changed from the base of the PR and between 5986b31 and d505862.

📒 Files selected for processing (7)
  • packages/genui/a2ui-playground/src/pages/AIChatPage.tsx
  • packages/genui/server/AGENTS.md
  • packages/genui/server/agent/a2ui-prompt.ts
  • packages/genui/server/agent/image-resolver.ts
  • packages/genui/server/app/a2ui/action/stream/route.ts
  • packages/genui/server/app/a2ui/stream/route.ts
  • packages/genui/server/service/a2ui-agent.ts

Comment thread packages/genui/a2ui-playground/src/pages/AIChatPage.tsx Outdated
Comment thread packages/genui/server/agent/image-resolver.ts Outdated
Comment thread packages/genui/server/agent/image-resolver.ts Outdated
@Sherry-hue Sherry-hue force-pushed the feat/genui-server-image branch 2 times, most recently from 03c5e0c to e914d71 Compare May 21, 2026 08:12
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (2)
packages/genui/server/agent/image-resolver.ts (2)

26-26: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Bound the process-wide image cache.

imageCache grows without eviction, so high query cardinality can cause unbounded memory growth over time.

Proposed fix
 const imageCache = new Map<string, Promise<string>>();
+const MAX_IMAGE_CACHE_ENTRIES = 1000;
@@
   if (!cached) {
     cached = resolvePexelsImage(query).then(
       (url) => url ?? picsumUrl(query),
       () => picsumUrl(query),
     );
+    if (imageCache.size >= MAX_IMAGE_CACHE_ENTRIES) {
+      const oldestKey = imageCache.keys().next().value;
+      if (oldestKey) imageCache.delete(oldestKey);
+    }
     imageCache.set(cacheKey, cached);
   }

Also applies to: 106-113

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/genui/server/agent/image-resolver.ts` at line 26, The global
imageCache (const imageCache = new Map<string, Promise<string>>()) is unbounded
and must be bounded to prevent memory growth; replace it with a size-limited
cache (either swap to an LRU implementation like the lru-cache package or
implement simple FIFO/LRU eviction) and ensure all accesses (the existing
imageCache.get / imageCache.set / imageCache.has usages around the resolver
functions) use the new bounded cache API or a wrapper helper (e.g.,
getCachedImage(key) / setCachedImage(key, promise)) that enforces eviction when
cache.size > MAX_IMAGE_CACHE_SIZE; pick a sensible MAX_IMAGE_CACHE_SIZE
constant, document it, and update any code paths that set entries (the sites
currently writing into imageCache) to use the wrapper or new cache instance.

126-128: ⚠️ Potential issue | 🔴 Critical | ⚡ Quick win

Add a timeout to the Pexels fetch.

The outbound fetch has no timeout/signal. A hung upstream can block this path and delay fallback behavior.

Proposed fix
 async function resolvePexelsImage(query: string): Promise<string | null> {
@@
-  const res = await fetch(url, {
-    headers: { Authorization: apiKey },
-  });
-  if (!res.ok) return null;
-
-  const data = await res.json() as PexelsSearchResponse;
-  const src = data.photos?.[0]?.src;
-  return src?.large2x ?? src?.large ?? src?.medium ?? src?.original ?? null;
+  const controller = new AbortController();
+  const timeout = setTimeout(() => controller.abort(), 5000);
+  try {
+    const res = await fetch(url, {
+      headers: { Authorization: apiKey },
+      signal: controller.signal,
+    });
+    if (!res.ok) return null;
+    const data = await res.json() as PexelsSearchResponse;
+    const src = data.photos?.[0]?.src;
+    return src?.large2x ?? src?.large ?? src?.medium ?? src?.original ?? null;
+  } catch {
+    return null;
+  } finally {
+    clearTimeout(timeout);
+  }
 }
#!/bin/bash
set -euo pipefail

FILE="packages/genui/server/agent/image-resolver.ts"

# Verify current fetch call has no signal/timeout control.
nl -ba "$FILE" | sed -n '117,140p'
rg -n "fetch\\(|AbortController|signal\\s*:" "$FILE"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/genui/server/agent/image-resolver.ts` around lines 126 - 128, The
fetch that assigns const res currently has no timeout; wrap it with an
AbortController: create a controller, start a setTimeout (e.g. 5000 ms) that
calls controller.abort(), pass controller.signal into the fetch call (preserving
headers including Authorization: apiKey), and clear the timeout once the fetch
completes; also ensure the surrounding async flow catches the abort/DOMException
and triggers the existing fallback path instead of hanging.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Duplicate comments:
In `@packages/genui/server/agent/image-resolver.ts`:
- Line 26: The global imageCache (const imageCache = new Map<string,
Promise<string>>()) is unbounded and must be bounded to prevent memory growth;
replace it with a size-limited cache (either swap to an LRU implementation like
the lru-cache package or implement simple FIFO/LRU eviction) and ensure all
accesses (the existing imageCache.get / imageCache.set / imageCache.has usages
around the resolver functions) use the new bounded cache API or a wrapper helper
(e.g., getCachedImage(key) / setCachedImage(key, promise)) that enforces
eviction when cache.size > MAX_IMAGE_CACHE_SIZE; pick a sensible
MAX_IMAGE_CACHE_SIZE constant, document it, and update any code paths that set
entries (the sites currently writing into imageCache) to use the wrapper or new
cache instance.
- Around line 126-128: The fetch that assigns const res currently has no
timeout; wrap it with an AbortController: create a controller, start a
setTimeout (e.g. 5000 ms) that calls controller.abort(), pass controller.signal
into the fetch call (preserving headers including Authorization: apiKey), and
clear the timeout once the fetch completes; also ensure the surrounding async
flow catches the abort/DOMException and triggers the existing fallback path
instead of hanging.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 28c2bc4e-6ddf-40d8-bd96-b0d4fac79a49

📥 Commits

Reviewing files that changed from the base of the PR and between d505862 and 03c5e0c.

📒 Files selected for processing (7)
  • packages/genui/a2ui-playground/src/pages/AIChatPage.tsx
  • packages/genui/server/AGENTS.md
  • packages/genui/server/agent/a2ui-prompt.ts
  • packages/genui/server/agent/image-resolver.ts
  • packages/genui/server/app/a2ui/action/stream/route.ts
  • packages/genui/server/app/a2ui/stream/route.ts
  • packages/genui/server/service/a2ui-agent.ts
✅ Files skipped from review due to trivial changes (1)
  • packages/genui/server/AGENTS.md

@Sherry-hue Sherry-hue force-pushed the feat/genui-server-image branch from e914d71 to abda536 Compare May 21, 2026 08:16
@codecov
Copy link
Copy Markdown

codecov Bot commented May 21, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

📢 Thoughts on this report? Let us know!

@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented May 21, 2026

Merging this PR will not alter performance

✅ 81 untouched benchmarks
⏩ 26 skipped benchmarks1


Comparing Sherry-hue:feat/genui-server-image (abda536) with main (5986b31)

Open in CodSpeed

Footnotes

  1. 26 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@relativeci
Copy link
Copy Markdown

relativeci Bot commented May 21, 2026

Web Explorer

#10116 Bundle Size — 903.53KiB (0%).

abda536(current) vs 5986b31 main#10113(baseline)

Bundle metrics  Change 1 change
                 Current
#10116
     Baseline
#10113
No change  Initial JS 45.06KiB 45.06KiB
No change  Initial CSS 2.22KiB 2.22KiB
No change  Cache Invalidation 0% 0%
No change  Chunks 9 9
No change  Assets 11 11
Change  Modules 230(-0.43%) 231
No change  Duplicate Modules 11 11
No change  Duplicate Code 27.12% 27.12%
No change  Packages 10 10
No change  Duplicate Packages 0 0
Bundle size by type  no changes
                 Current
#10116
     Baseline
#10113
No change  JS 499.15KiB 499.15KiB
No change  Other 402.16KiB 402.16KiB
No change  CSS 2.22KiB 2.22KiB

Bundle analysis reportBranch Sherry-hue:feat/genui-server-ima...Project dashboard


Generated by RelativeCIDocumentationReport issue

@relativeci
Copy link
Copy Markdown

relativeci Bot commented May 21, 2026

React MTF Example

#1675 Bundle Size — 208.75KiB (0%).

abda536(current) vs 5986b31 main#1672(baseline)

Bundle metrics  no changes
                 Current
#1675
     Baseline
#1672
No change  Initial JS 0B 0B
No change  Initial CSS 0B 0B
No change  Cache Invalidation 0% 0%
No change  Chunks 0 0
No change  Assets 3 3
No change  Modules 195 195
No change  Duplicate Modules 77 77
No change  Duplicate Code 44.17% 44.17%
No change  Packages 2 2
No change  Duplicate Packages 0 0
Bundle size by type  no changes
                 Current
#1675
     Baseline
#1672
No change  IMG 111.23KiB 111.23KiB
No change  Other 97.52KiB 97.52KiB

Bundle analysis reportBranch Sherry-hue:feat/genui-server-ima...Project dashboard


Generated by RelativeCIDocumentationReport issue

@relativeci
Copy link
Copy Markdown

relativeci Bot commented May 21, 2026

React Example with Element Template

#811 Bundle Size — 202.16KiB (0%).

abda536(current) vs 5986b31 main#808(baseline)

Bundle metrics  no changes
                 Current
#811
     Baseline
#808
No change  Initial JS 0B 0B
No change  Initial CSS 0B 0B
No change  Cache Invalidation 0% 0%
No change  Chunks 0 0
No change  Assets 4 4
No change  Modules 100 100
No change  Duplicate Modules 30 30
No change  Duplicate Code 39.22% 39.22%
No change  Packages 2 2
No change  Duplicate Packages 0 0
Bundle size by type  no changes
                 Current
#811
     Baseline
#808
No change  IMG 145.76KiB 145.76KiB
No change  Other 56.41KiB 56.41KiB

Bundle analysis reportBranch Sherry-hue:feat/genui-server-ima...Project dashboard


Generated by RelativeCIDocumentationReport issue

@relativeci
Copy link
Copy Markdown

relativeci Bot commented May 21, 2026

React External

#1657 Bundle Size — 698.01KiB (0%).

abda536(current) vs 5986b31 main#1654(baseline)

Bundle metrics  no changes
                 Current
#1657
     Baseline
#1654
No change  Initial JS 0B 0B
No change  Initial CSS 0B 0B
No change  Cache Invalidation 0% 0%
No change  Chunks 0 0
No change  Assets 3 3
No change  Modules 17 17
No change  Duplicate Modules 5 5
No change  Duplicate Code 8.59% 8.59%
No change  Packages 0 0
No change  Duplicate Packages 0 0
Bundle size by type  no changes
                 Current
#1657
     Baseline
#1654
No change  Other 698.01KiB 698.01KiB

Bundle analysis reportBranch Sherry-hue:feat/genui-server-ima...Project dashboard


Generated by RelativeCIDocumentationReport issue

@relativeci
Copy link
Copy Markdown

relativeci Bot commented May 21, 2026

React Example

#8542 Bundle Size — 237.81KiB (0%).

abda536(current) vs 5986b31 main#8539(baseline)

Bundle metrics  no changes
                 Current
#8542
     Baseline
#8539
No change  Initial JS 0B 0B
No change  Initial CSS 0B 0B
No change  Cache Invalidation 0% 0%
No change  Chunks 0 0
No change  Assets 4 4
No change  Modules 200 200
No change  Duplicate Modules 80 80
No change  Duplicate Code 44.68% 44.68%
No change  Packages 2 2
No change  Duplicate Packages 0 0
Bundle size by type  no changes
                 Current
#8542
     Baseline
#8539
No change  IMG 145.76KiB 145.76KiB
No change  Other 92.05KiB 92.05KiB

Bundle analysis reportBranch Sherry-hue:feat/genui-server-ima...Project dashboard


Generated by RelativeCIDocumentationReport issue

@Sherry-hue Sherry-hue merged commit 8083591 into lynx-family:main May 21, 2026
83 of 85 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants