Skip to content
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
8054016
feat: overlay persisted image-region boxes on source figures
seonghobae Aug 20, 2026
e7d1b35
docs: avoid image overlay ADR collision
seonghobae Aug 20, 2026
6681688
test: cover invalid image overlay bounds
seonghobae Aug 20, 2026
e6fd907
docs: allocate unique image overlay ADR number
seonghobae Aug 20, 2026
a3b1ec0
Merge remote-tracking branch 'origin/fix/buyer-image-evidence' into c…
seonghobae Aug 20, 2026
2da6a18
fix: validate embedded image sources at render boundary
seonghobae Aug 20, 2026
9c2d26c
Merge remote-tracking branch 'origin/fix/buyer-image-evidence' into f…
seonghobae Aug 20, 2026
66fc696
Merge remote-tracking branch 'origin/fix/buyer-image-evidence' into f…
seonghobae Aug 20, 2026
d8a938c
Merge remote-tracking branch 'origin/fix/buyer-image-evidence' into f…
seonghobae Aug 20, 2026
7504774
Merge remote-tracking branch 'origin/fix/buyer-image-evidence' into f…
seonghobae Aug 21, 2026
381098a
Merge latest buyer image evidence into region overlays
seonghobae Aug 21, 2026
6c8f31c
Merge commit 'ff562a99cf57759b61ee657293b74ea3938cf1ac' into HEAD
seonghobae Aug 21, 2026
bd3f649
Merge remote-tracking branch 'refs/remotes/origin/feat/buyer-image-re…
seonghobae Aug 21, 2026
f44ec17
Merge remote-tracking branch 'origin/fix/buyer-image-evidence' into c…
seonghobae Aug 21, 2026
76dc329
Merge remote-tracking branch 'origin/feat/buyer-image-region-overlays…
seonghobae Aug 21, 2026
6d716a1
Merge commit 'b818ef88ef4a5964ddeb2752b610170feab0d2cd' into HEAD
seonghobae Aug 21, 2026
a550f8f
fix: preserve buyer image region evidence
seonghobae Aug 21, 2026
05860e9
Merge remote-tracking branch 'origin/feat/buyer-image-region-overlays…
seonghobae Aug 21, 2026
5336b28
fix: preserve image region evidence during restack
seonghobae Aug 21, 2026
324f1fa
Merge remote-tracking branch 'origin/feat/buyer-image-region-overlays…
seonghobae Aug 21, 2026
c9b39eb
Merge commit '3131868a9ffbd85fb32895eaebf0a130dd6a9af7' into HEAD
seonghobae Aug 21, 2026
8b37682
fix: preserve buyer chrome across image overlays
seonghobae Aug 21, 2026
519a8c3
fix: keep evidence panel above buyer header
seonghobae Aug 21, 2026
9339e2b
Merge branch 'fix/buyer-image-evidence' of https://github.com/Context…
seonghobae Aug 21, 2026
57e75d5
fix: preserve image region evidence state
seonghobae Aug 21, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,10 @@ contextual-orchestrator owns model discovery and selection.
retaining the original asset and provenance. Recognize image DOM/visual
regions before OCR, descriptions, Keyman extraction, or embeddings. Store
region-level evidence; never show an internal LLM instruction such as
`This post is an image` to a buyer.
`This post is an image` to a buyer. When a source image can be reattached,
draw only persisted visual-region boxes; never invent coordinates. If the
bitmap is missing or a box is invalid, keep the region list and do not
fabricate a locator (ADR 0110 / 0113).

## Source parsing and semantic units

Expand All @@ -150,7 +153,8 @@ contextual-orchestrator owns model discovery and selection.
paragraphs, not the authoring application's spacing workaround.
- Image descriptions, OCR text, and region evidence are analysis artifacts,
not buyer-facing prompt instructions. Buyer UI shows the source content and
useful captions/evidence only, with provenance where appropriate.
useful captions/evidence only, with provenance where appropriate. Overlay
boxes use stored ratios; a missing image fails closed to the list.

## Pluggable channels: never fake a missing signal

Expand Down
17 changes: 17 additions & 0 deletions CHANGELOG.d/2.13.2-image-region-overlays.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# 2.13.2 — Overlay persisted image-region boxes

## Added

- Buyer post figures now draw persisted visual-region boxes on a reattached
source image. Selecting a box announces that region's caption or OCR.
- Invalid, missing, or out-of-bounds coordinates stay list-only. When the
bitmap cannot be reattached, the region list remains the fallback and no
locator is invented.

## Changed

- Image-region overlay and current-region labels are translated through the
five-locale catalog (Korean, English fallback, Chinese, Japanese,
Vietnamese).
- Buyer image rendering now rejects script, SVG, external, and malformed
source URLs before they reach an image element.
32 changes: 32 additions & 0 deletions docs/adr/0113-buyer-image-region-overlays.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# ADR 0113: Overlay persisted visual-region boxes on the source image

- Status: Accepted
- Date: 2026-08-20
- Depends on: [0067](0067-visual-region-vision-agent.md), [0091](0091-visual-region-embedding-persistence.md), [0110](0110-buyer-image-evidence-rendering.md)

## Context

ADR 0067 and ADR 0091 persist parent-relative `x_ratio`, `y_ratio`,
`width_ratio`, and `height_ratio` for each accepted visual region. ADR 0110
renders captions, OCR, tags, and a region list, but the buyer still cannot
see which panel of the source image supplied that evidence.

## Decision

- When the source image can be reattached and a region has a finite persisted
box inside the unit square, draw that box on the figure as a
keyboard-focusable control. Selecting it announces the persisted caption or
OCR as the current image region.
- Never invent coordinates. Invalid, missing, non-finite, zero-area, or
out-of-bounds boxes stay list-only. When the source image cannot be
reattached, keep the region list and do not draw empty overlays.
- Do not surface LLM instructions from image `unit_text`. Overlay labels come
only from persisted region caption or extracted text.
- Translate overlay and current-region labels through the five-locale UI
catalog.

## Consequences

Buyers can locate panel-level evidence on the source image. Search and
embedding remain bound to the existing region tables. A missing bitmap still
fails closed to the list rather than fabricating a locator.
2 changes: 1 addition & 1 deletion frontend/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "frontend",
"private": true,
"version": "2.12.6",
"version": "2.13.2",
"type": "module",
"scripts": {
"dev": "vite",
Expand Down
85 changes: 85 additions & 0 deletions frontend/src/App.css

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Wholesale App.css rewrite removes classes still referenced by components

This diff replaces the entire App.css with a different design system (token vars like var(--surface) swapped for canvas/canvastext, and shell/login/footer/button classes deleted). Classes still referenced in TSX — e.g. app-shell, login-card, app-footer, btn-primary, btn-secondary, app-header-title — no longer have rules here, and modal z-index declarations on .popup-backdrop/.popup-panel were dropped. This is far outside the stated overlay scope and is most likely inherited from the stacked parent (#307) rather than introduced by this PR, so it is not flagged as a bug, but a reviewer confirming the exact base-to-head diff should verify these removals are intentional and that login/footer/button styling and modal stacking are not regressed.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Original file line number Diff line number Diff line change
Expand Up @@ -364,6 +364,91 @@
height: auto;
}

.post-embedded-image-frame {
position: relative;
display: inline-block;
max-width: 100%;
}

.post-embedded-image-frame img {
display: block;
max-width: 100%;
height: auto;
}

.post-image-region-overlays {
position: absolute;
inset: 0;
}
Comment thread
devin-ai-integration[bot] marked this conversation as resolved.
Comment on lines +514 to +530

@devin-ai-integration devin-ai-integration Bot Aug 21, 2026 •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Overlay percentages assume the frame tightly wraps the bitmap

Overlay boxes at PostBody.tsx position via percentage offsets relative to .post-image-region-overlays (position:absolute; inset:0), which fills .post-embedded-image-frame (display:inline-block, max-width:100%). This only aligns with the visible bitmap if the frame shrinks exactly to the rendered image box. The <img> uses max-width:100%; height:auto, so the inline-block frame should size to the image — but if the image intrinsic width is unknown before load, or the frame ever gains padding/border, the percentage-anchored boxes would drift from the actual panels. Worth a visual check with a real (non-1x1) bitmap.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.


.post-image-region-overlay {
position: absolute;
box-sizing: border-box;
margin: 0;
padding: 0;
border: 2px solid var(--color-accent-info);
background: var(--color-accent-info-background);
cursor: pointer;
appearance: none;
}

.post-image-region-overlay[aria-pressed="true"] {
border-color: var(--color-accent);
background: var(--color-accent-background);
}

.post-image-region-overlay:focus-visible {
outline: 2px solid var(--color-accent);
outline-offset: 2px;
}

.post-image-current-region {
margin: 0.4rem 0 0;
font-size: 0.85rem;
color: var(--text-h);
}

@media (prefers-reduced-motion: no-preference) {
.post-image-region-overlay {
transition: border-color 160ms ease, background-color 160ms ease;
}
}

@media print {
body * {
visibility: hidden;
}

.popup-backdrop,
.popup-panel,
.popup-panel * {
visibility: visible;
}

.popup-backdrop {
position: absolute;
inset: 0;
display: block;
background: none;
}

.popup-panel {
position: static;
width: auto;
max-width: none;
max-height: none;
overflow: visible;
padding: 0;
border-radius: 0;
}

.popup-close,
.post-actions,
.post-action-status {
display: none !important;
}
}

Comment on lines +571 to +604

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Printing any page without an open post popup yields a blank sheet

The new @media print block hides body * and only re-reveals .popup-backdrop/.popup-panel. With no post popup open, printing a board, calendar, lineage, or report page now outputs nothing; previously the full page printed.

Prompt for agents
The @media print block in frontend/src/App.css sets `body * { visibility: hidden; }` and only makes the popup backdrop/panel visible again. This means printing any view where a post popup is NOT open (board list, calendar, lineage graph, reports) produces a blank page, which is a regression from the prior behavior (no print rules, so full page printed). If the intent is only to print the open post popup, scope the visibility reset so it applies only when a popup is present (for example, gate it behind a body-level class toggled when the modal is open, or narrow the selector) so that normal pages remain printable when no popup is open.
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

.post-embedded-image figcaption {
margin-top: 0.4rem;
font-size: 0.85rem;
Expand Down
177 changes: 177 additions & 0 deletions frontend/src/PostBody.test.tsx
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { render, screen } from "@testing-library/react";
import userEvent from "@testing-library/user-event";
import { describe, expect, it } from "vitest";
import { PostBody } from "./PostBody";

Expand Down Expand Up @@ -271,6 +272,182 @@ describe("PostBody", () => {
expect(screen.getByText("diagram, process")).toBeInTheDocument();
expect(screen.getByText("Main panel")).toBeInTheDocument();
expect(screen.queryByText(/This post is an image/)).not.toBeInTheDocument();
expect(screen.queryByRole("button", { name: /Main panel/ })).not.toBeInTheDocument();
});

it("overlays persisted region boxes on a reattached source image", async () => {
const user = userEvent.setup();
const source =
'<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=" />';

render(
<PostBody
body={source}
structureUnits={[
{
unit_index: 0,
unit_kind_code: "image",
unit_label: "img",
unit_text: "This post is an image. Ask questions to read its text.",
indent_level: 0,
indent_source_code: "unresolved",
indent_confidence: 0,
indent_evidence: "",
},
]}
imageContent={[
{
unit_index: 0,
mime_type: "image/png",
status_code: "described",
extracted_text: "Visible OCR",
caption: "A process diagram",
tags: ["diagram"],
regions: [
{
region_index: 0,
x_ratio: 0.1,
y_ratio: 0.2,
width_ratio: 0.3,
height_ratio: 0.4,
status_code: "described",
extracted_text: "Region OCR",
caption: "Main panel",
tags: ["panel"],
},
{
region_index: 1,
x_ratio: 0.9,
y_ratio: 0.9,
width_ratio: 0.5,
height_ratio: 0.5,
status_code: "described",
extracted_text: "Invented box must not render",
caption: "Overflow panel",
tags: [],
},
{
region_index: 2,
x_ratio: 0.7,
y_ratio: 0.6,
width_ratio: 0.30000000000000004,
height_ratio: 0.4,
status_code: "described",
extracted_text: "Edge region OCR",
caption: "Edge panel",
tags: [],
},
],
},
]}
/>,
);

const overlay = screen.getByRole("button", { name: "Image region: Main panel" });
expect(overlay).toHaveStyle({
left: "10%",
top: "20%",
width: "30%",
height: "40%",
});
expect(screen.getByRole("button", { name: "Image region: Edge panel" })).toBeInTheDocument();
expect(screen.queryByRole("button", { name: /Overflow panel/ })).not.toBeInTheDocument();
expect(screen.queryByText(/This post is an image/)).not.toBeInTheDocument();
expect(screen.queryByText(/Invented box must not render/)).not.toBeInTheDocument();
expect(screen.getByText("Overflow panel")).toBeInTheDocument();

overlay.focus();
expect(overlay).toHaveFocus();
await user.click(overlay);
expect(screen.getByText("Current image region: Main panel")).toBeInTheDocument();
expect(overlay).toHaveAttribute("aria-pressed", "true");
await user.click(overlay);
expect(screen.queryByText("Current image region: Main panel")).not.toBeInTheDocument();
expect(overlay).toHaveAttribute("aria-pressed", "false");
});

it("keeps invalid persisted boxes list-only and does not create overlay controls", () => {
render(
<PostBody
body={'<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=" />'}
structureUnits={[
{
unit_index: 0,
unit_kind_code: "image",
unit_label: "img",
unit_text: "internal image instruction",
indent_level: 0,
indent_source_code: "unresolved",
indent_confidence: 0,
indent_evidence: "",
},
]}
imageContent={[
{
unit_index: 0,
mime_type: "image/png",
status_code: "described",
extracted_text: "",
caption: "",
tags: [],
regions: [
{
region_index: 0,
x_ratio: Number.NaN,
y_ratio: 0,
width_ratio: 0.2,
height_ratio: 0.2,
status_code: "unavailable",
extracted_text: "NaN box",
caption: "NaN box",
tags: [],
},
{
region_index: 1,
x_ratio: -0.1,
y_ratio: 0,
width_ratio: 0.2,
height_ratio: 0.2,
status_code: "unavailable",
extracted_text: "Negative box",
caption: "Negative box",
tags: [],
},
{
region_index: 2,
x_ratio: 0,
y_ratio: 0,
width_ratio: 0,
height_ratio: 0.2,
status_code: "unavailable",
extracted_text: "Zero box",
caption: "Zero box",
tags: [],
},
{
region_index: 3,
x_ratio: 0.8,
y_ratio: 0.8,
width_ratio: 0.3,
height_ratio: 0.2,
status_code: "unavailable",
extracted_text: "Overflow box",
caption: "Overflow box",
tags: [],
},
],
},
]}
/>,
);

expect(screen.queryByRole("group", { name: "Image regions" })).not.toBeInTheDocument();
expect(screen.queryByRole("button", { name: /Image region/ })).not.toBeInTheDocument();
expect(screen.getByText("NaN box")).toBeInTheDocument();
expect(screen.getByText("Negative box")).toBeInTheDocument();
expect(screen.getByText("Zero box")).toBeInTheDocument();
expect(screen.getByText("Overflow box")).toBeInTheDocument();
expect(screen.getByAltText("Embedded image")).toBeInTheDocument();
});

it("renders pipe-delimited image OCR as a buyer-facing table", () => {
Expand Down
Loading
Loading