Skip to content

Agentic Review: Show banner when review is stale#34981

Merged
ghengeveld merged 10 commits into
yann/agentic-review-mcp-integrationfrom
stale-review-banner
Jun 4, 2026
Merged

Agentic Review: Show banner when review is stale#34981
ghengeveld merged 10 commits into
yann/agentic-review-mcp-integrationfrom
stale-review-banner

Conversation

@ghengeveld

@ghengeveld ghengeveld commented May 29, 2026

Copy link
Copy Markdown
Member

What I did

Adds staleness detection for agentic reviews. When a review is pushed (via @storybook/addon-mcp), the dev server now subscribes to source-file change events from core's change detection. If any watched source file changes after the review was created (past a 1s grace window that absorbs the agent's own edits, which land just after the review is cached), the review is marked stale and a banner — "New changes were made. This review may be stale." — is shown on both the summary and details screens. Staleness rides on the cached review state, so tabs that open after the change still see the banner.

Implementation:

  • Core: a process-wide subscribeToSourceFileChanges notifier (source-changes.ts), fired from StoryDependencyGraphService on each file-change event and re-exported as experimental_subscribeToSourceFileChanges.
  • addon-review: the server-side preset subscribes and emits a REVIEW_STALE event; ReviewState gains a stale flag; a new StaleBanner component plus DetailsScreen/SummaryScreen/ReviewPage wiring surface it to users.
Screenshot 2026-05-30 at 00 57 31

Checklist for Contributors

Testing

The changes in this PR are covered in the following automated tests:

  • stories
  • unit tests
  • integration tests
  • end-to-end tests

Manual testing

Caution

This section is mandatory for all contributions. If you believe no manual test is necessary, please state so explicitly. Thanks!

  1. Run the internal Storybook UI: cd code && yarn storybook:ui
  2. Push a review through the MCP / agentic-review flow so a review is displayed (banner-free).
  3. Edit and save any watched source file.
  4. Confirm the stale banner ("New changes were made. This review may be stale.") appears on both the summary and details screens, and that it persists when reopening/reconnecting a tab.

Documentation

  • Add or update documentation reflecting your changes
  • If you are deprecating/removing a feature, make sure to update
    MIGRATION.MD

Checklist for Maintainers

  • When this PR is ready for testing, make sure to add ci:normal, ci:merged or ci:daily GH label to it to run a specific set of sandboxes. The particular set of sandboxes can be found in code/lib/cli-storybook/src/sandbox-templates.ts

  • Make sure this PR contains one of the labels below:

    Available labels
    • bug: Internal changes that fixes incorrect behavior.
    • maintenance: User-facing maintenance tasks.
    • dependencies: Upgrading (sometimes downgrading) dependencies.
    • build: Internal-facing build tooling & test updates. Will not show up in release changelog.
    • cleanup: Minor cleanup style change. Will not show up in release changelog.
    • documentation: Documentation only changes. Will not show up in release changelog.
    • feature request: Introducing a new feature.
    • BREAKING CHANGE: Changes that break compatibility in some way with current major version.
    • other: Changes that don't fit in the above categories.

🦋 Canary release

This PR does not have a canary release associated. You can request a canary release of this pull request by mentioning the @storybookjs/core team here.

core team members can create a canary release here or locally with gh workflow run --repo storybookjs/storybook publish.yml --field pr=<PR_NUMBER>

Summary by CodeRabbit

  • New Features

    • Reviews now show a prominent "New changes were made. This review may be stale." banner.
    • Staleness is surfaced across review summary and details; cached reviews replay as stale when source files change (with a short grace window).
    • Storybook stories added to demonstrate the stale state in the UI.
  • Tests

    • Added comprehensive tests covering staleness detection, source-change notifications, replay/ignore behaviors, and resilience of notification delivery.

@ghengeveld ghengeveld self-assigned this May 29, 2026
@coderabbitai

coderabbitai Bot commented May 29, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

A staleness detection flow: core source-change notifications, server-side grace-window staleness marking that emits EVENTS.REVIEW_STALE and stores ReviewState.stale, a StaleBanner UI, and ReviewPage wiring that passes isStale to DetailsScreen and SummaryScreen.

Changes

Review staleness detection

Layer / File(s) Summary
Core file-change notification API
code/core/src/core-server/change-detection/source-changes.ts, code/core/src/core-server/change-detection/source-changes.test.ts, code/core/src/core-server/change-detection/index.ts, code/core/src/core-server/change-detection/StoryDependencyGraphService.ts, code/core/src/core-server/index.ts
Adds a process-level listener registry for FileChangeEvent subscriptions with fanout, unsubscribe, error isolation, and test-reset. Tests verify subscriber behavior. Notifies subscribers from StoryDependencyGraphService and re-exports the API as experimental_subscribeToSourceFileChanges.
Review state model and events
code/addons/review/src/review-state.ts, code/addons/review/src/constants.ts
Extends ReviewState with optional stale?: boolean. Adds REVIEW_STALE event constant to signal staleness to clients.
Server-side staleness detection
code/addons/review/src/preset.ts, code/addons/review/src/preset.test.ts
Preset subscribes to source changes (configurable via ServerChannelOptions), applies STALE_GRACE_MS relative to cached review createdAt, marks cached review stale and emits REVIEW_STALE after the window. Tests cover grace-window boundaries, single emission, clearing on push, and replay.
Stale banner component
code/addons/review/src/components/StaleBanner.tsx
New React component: themed horizontal bar displaying "New changes were made. This review may be stale." with role="status" and aria-live="polite".
Screen integration
code/addons/review/src/screens/DetailsScreen.tsx, DetailsScreen.stories.tsx, SummaryScreen.tsx, SummaryScreen.stories.tsx
DetailsScreen and SummaryScreen accept optional isStale?: boolean and conditionally render StaleBanner at the top. Stories assert banner visibility when flag is true.
ReviewPage state tracking
code/addons/review/src/ReviewPage.ts
Adds local isStale state, sets it from DISPLAY_REVIEW (reads next.stale) and from REVIEW_STALE channel events, and forwards isStale to both screen components.
sequenceDiagram
  participant StoryDependencyGraphService
  participant SourceChangeRegistry
  participant ReviewPreset
  participant ReviewPage
  participant DetailsScreen
  participant SummaryScreen
  StoryDependencyGraphService->>SourceChangeRegistry: notifySourceFileChange(FileChangeEvent)
  SourceChangeRegistry->>ReviewPreset: subscribed listener(FileChangeEvent)
  ReviewPreset->>ReviewPage: emit(EVENTS.REVIEW_STALE)
  ReviewPage->>DetailsScreen: prop isStale=true
  ReviewPage->>SummaryScreen: prop isStale=true
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • storybookjs/storybook#35009: Adds notifySourceFileChange emission in StoryDependencyGraphService.handleFileChange, which is the core signal this PR's staleness detection consumes via subscribeToSourceFileChanges.

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
code/core/src/core-server/change-detection/source-changes.ts (1)

30-38: ⚡ Quick win

Consider logging swallowed listener errors.

Isolating listener failures is correct, but a fully silent catch makes a misbehaving subscriber undiagnosable. A debug-level log keeps the isolation guarantee while preserving observability.

♻️ Proposed change
+import { logger } from 'storybook/internal/node-logger';
 import type { FileChangeEvent } from './adapters/index.ts';
 export function notifySourceFileChange(event: FileChangeEvent): void {
   for (const listener of listeners) {
     try {
       listener(event);
-    } catch {
-      // A listener failure must never break change detection.
+    } catch (error) {
+      // A listener failure must never break change detection.
+      logger.debug(
+        `Source-file change listener threw: ${error instanceof Error ? error.message : String(error)}`
+      );
     }
   }
 }

As per coding guidelines: "Use storybook/internal/node-logger for server-side logging instead of raw console.*".

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@code/core/src/core-server/change-detection/source-changes.ts` around lines 30
- 38, The notifySourceFileChange function currently swallows all listener errors
silently; update it to catch the error and emit a debug-level log using
storybook/internal/node-logger so failures are observable without breaking
change detection: import the logger, then inside notifySourceFileChange's catch
block (for the listeners iteration) call logger.debug or logger.error with a
concise message including the listener identifier (if available) and the caught
error/stack to aid diagnosis while preserving the isolation guarantee.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@code/core/src/core-server/change-detection/source-changes.ts`:
- Around line 30-38: The notifySourceFileChange function currently swallows all
listener errors silently; update it to catch the error and emit a debug-level
log using storybook/internal/node-logger so failures are observable without
breaking change detection: import the logger, then inside
notifySourceFileChange's catch block (for the listeners iteration) call
logger.debug or logger.error with a concise message including the listener
identifier (if available) and the caught error/stack to aid diagnosis while
preserving the isolation guarantee.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: e3c8373d-48df-4f57-a554-21f5371e61dd

📥 Commits

Reviewing files that changed from the base of the PR and between 31a2813 and d3fdf3d.

📒 Files selected for processing (15)
  • code/addons/review/src/ReviewPage.ts
  • code/addons/review/src/components/StaleBanner.tsx
  • code/addons/review/src/constants.ts
  • code/addons/review/src/preset.test.ts
  • code/addons/review/src/preset.ts
  • code/addons/review/src/review-state.ts
  • code/addons/review/src/screens/DetailsScreen.stories.tsx
  • code/addons/review/src/screens/DetailsScreen.tsx
  • code/addons/review/src/screens/SummaryScreen.stories.tsx
  • code/addons/review/src/screens/SummaryScreen.tsx
  • code/core/src/core-server/change-detection/ChangeDetectionService.ts
  • code/core/src/core-server/change-detection/index.ts
  • code/core/src/core-server/change-detection/source-changes.test.ts
  • code/core/src/core-server/change-detection/source-changes.ts
  • code/core/src/core-server/index.ts

@ghengeveld ghengeveld added feature request ci:normal Run our default set of CI jobs (choose this for most PRs). qa:needed Pull Requests that will need manual QA prior to release. labels Jun 2, 2026
@ghengeveld ghengeveld requested a review from yannbf June 2, 2026 14:14
@storybook-app-bot

storybook-app-bot Bot commented Jun 2, 2026

Copy link
Copy Markdown

Package Benchmarks

Commit: 1d1490c, ran on 4 June 2026 at 14:55:08 UTC

No significant changes detected, all good. 👏

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@code/core/src/core-server/change-detection/StoryDependencyGraphService.ts`:
- Around line 388-390: The call to notifySourceFileChange(event) in
handleFileChange is currently emitted for both real adapter filesystem events
and synthetic add/unlink replays from refreshStoryFiles, causing internal
reconciliations to be treated as external changes; update handleFileChange (and
its callers in refreshStoryFiles) to only call notifySourceFileChange when the
event is adapter-originated (e.g., check an origin/type property on the event)
or add a suppressNotification boolean parameter to handleFileChange that
refreshStoryFiles passes true for, and ensure notifySourceFileChange(event) is
skipped when suppressNotification is true so only real adapter events trigger
external notifications.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 3d4a1d62-7f1e-47fa-9238-8372f8d4162a

📥 Commits

Reviewing files that changed from the base of the PR and between 700b464 and a27fccc.

📒 Files selected for processing (4)
  • code/addons/review/src/screens/SummaryScreen.tsx
  • code/core/src/core-server/change-detection/StoryDependencyGraphService.ts
  • code/core/src/core-server/change-detection/index.ts
  • code/core/src/core-server/index.ts
✅ Files skipped from review due to trivial changes (1)
  • code/core/src/core-server/index.ts
🚧 Files skipped from review as they are similar to previous changes (2)
  • code/core/src/core-server/change-detection/index.ts
  • code/addons/review/src/screens/SummaryScreen.tsx

Comment on lines +388 to +390
// Surface the raw change to external subscribers (e.g. addon-review's
// staleness check) before patching — they only care that a file changed.
notifySourceFileChange(event);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Only notify external subscribers for adapter-originated file changes.

handleFileChange() is also used for the synthetic add/unlink replays queued by refreshStoryFiles() on Lines 314-322. Emitting notifySourceFileChange(event) here makes those internal reconciliation patches look like fresh filesystem events, so a review created after the real change can still be marked stale by the later replay. Please gate the notification to adapter-originated events only, or add a flag to suppress it for internal replays.

Suggested fix
-  private async handleFileChange(event: FileChangeEvent): Promise<void> {
+  private async handleFileChange(
+    event: FileChangeEvent,
+    notifyExternalSubscribers = true
+  ): Promise<void> {
     if (this.disposed || !this.incrementalPatcher) {
       return;
     }
-    // Surface the raw change to external subscribers (e.g. addon-review's
-    // staleness check) before patching — they only care that a file changed.
-    notifySourceFileChange(event);
+    if (notifyExternalSubscribers) {
+      // Surface raw adapter events to external subscribers (e.g. addon-review's
+      // staleness check) before patching — they only care that a file changed.
+      notifySourceFileChange(event);
+    }
     try {
       await this.incrementalPatcher.patch(event);
     } catch (error) {
       logger.warn(
         `Change detection: failed to apply ${event.kind} for ${event.path}: ${error instanceof Error ? error.message : String(error)}`
@@
     for (const path of added) {
       this.patchQueue = this.patchQueue
-        .then(() => this.handleFileChange({ kind: 'add', path }))
+        .then(() => this.handleFileChange({ kind: 'add', path }, false))
         .catch(() => undefined);
     }
     for (const path of removed) {
       this.patchQueue = this.patchQueue
-        .then(() => this.handleFileChange({ kind: 'unlink', path }))
+        .then(() => this.handleFileChange({ kind: 'unlink', path }, false))
         .catch(() => undefined);
     }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@code/core/src/core-server/change-detection/StoryDependencyGraphService.ts`
around lines 388 - 390, The call to notifySourceFileChange(event) in
handleFileChange is currently emitted for both real adapter filesystem events
and synthetic add/unlink replays from refreshStoryFiles, causing internal
reconciliations to be treated as external changes; update handleFileChange (and
its callers in refreshStoryFiles) to only call notifySourceFileChange when the
event is adapter-originated (e.g., check an origin/type property on the event)
or add a suppressNotification boolean parameter to handleFileChange that
refreshStoryFiles passes true for, and ensure notifySourceFileChange(event) is
skipped when suppressNotification is true so only real adapter events trigger
external notifications.

ghengeveld and others added 2 commits June 3, 2026 00:05
…ion' into stale-review-banner

Co-authored-by: Cursor <cursoragent@cursor.com>
Adds a debug-level log in notifySourceFileChange's catch so a misbehaving
subscriber is diagnosable while preserving the isolation guarantee. Also
corrects the doc reference to StoryDependencyGraphService (the publisher
after the dependency-graph refactor).

Co-authored-by: Cursor <cursoragent@cursor.com>
@ghengeveld ghengeveld force-pushed the stale-review-banner branch from 138c0d8 to 40e1c24 Compare June 2, 2026 22:06

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
code/core/src/core-server/change-detection/StoryDependencyGraphService.ts (1)

388-390: ⚠️ Potential issue | 🟠 Major | 🏗️ Heavy lift

Internal reconciliation replays still trigger external staleness notifications.

notifySourceFileChange(event) fires for every call to handleFileChange, including the synthetic add/unlink replays enqueued by refreshStoryFiles() (Lines 314-322). Those replays are internal story-index reconciliations, not fresh filesystem events, so a review created after the original change can still be marked stale by a later replay. Gate the notification to adapter-originated events only.

Suggested fix
-  private async handleFileChange(event: FileChangeEvent): Promise<void> {
+  private async handleFileChange(
+    event: FileChangeEvent,
+    notifyExternalSubscribers = true
+  ): Promise<void> {
     if (this.disposed || !this.incrementalPatcher) {
       return;
     }
-    // Surface the raw change to external subscribers (e.g. addon-review's
-    // staleness check) before patching — they only care that a file changed.
-    notifySourceFileChange(event);
+    if (notifyExternalSubscribers) {
+      // Surface raw adapter events to external subscribers (e.g. addon-review's
+      // staleness check) before patching — they only care that a file changed.
+      notifySourceFileChange(event);
+    }

And update the refreshStoryFiles callers:

     for (const path of added) {
       this.patchQueue = this.patchQueue
-        .then(() => this.handleFileChange({ kind: 'add', path }))
+        .then(() => this.handleFileChange({ kind: 'add', path }, false))
         .catch(() => undefined);
     }
     for (const path of removed) {
       this.patchQueue = this.patchQueue
-        .then(() => this.handleFileChange({ kind: 'unlink', path }))
+        .then(() => this.handleFileChange({ kind: 'unlink', path }, false))
         .catch(() => undefined);
     }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@code/core/src/core-server/change-detection/StoryDependencyGraphService.ts`
around lines 388 - 390, handleFileChange currently calls
notifySourceFileChange(event) for every event, which causes synthetic replays
from refreshStoryFiles to trigger external staleness notifications; change
handleFileChange to only call notifySourceFileChange for adapter-originated
events (e.g., check a unique marker like event.origin === 'adapter' or
event.isSynthetic === false) and ensure refreshStoryFiles enqueues/creates
replay events with that marker set (e.g., event.origin = 'internal' or
event.isSynthetic = true) so those synthetic add/unlink replays do not trigger
notifySourceFileChange.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Duplicate comments:
In `@code/core/src/core-server/change-detection/StoryDependencyGraphService.ts`:
- Around line 388-390: handleFileChange currently calls
notifySourceFileChange(event) for every event, which causes synthetic replays
from refreshStoryFiles to trigger external staleness notifications; change
handleFileChange to only call notifySourceFileChange for adapter-originated
events (e.g., check a unique marker like event.origin === 'adapter' or
event.isSynthetic === false) and ensure refreshStoryFiles enqueues/creates
replay events with that marker set (e.g., event.origin = 'internal' or
event.isSynthetic = true) so those synthetic add/unlink replays do not trigger
notifySourceFileChange.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 61a9cabb-4675-4253-8f5f-adb930228c9b

📥 Commits

Reviewing files that changed from the base of the PR and between 138c0d8 and 40e1c24.

📒 Files selected for processing (15)
  • code/addons/review/src/ReviewPage.ts
  • code/addons/review/src/components/StaleBanner.tsx
  • code/addons/review/src/constants.ts
  • code/addons/review/src/preset.test.ts
  • code/addons/review/src/preset.ts
  • code/addons/review/src/review-state.ts
  • code/addons/review/src/screens/DetailsScreen.stories.tsx
  • code/addons/review/src/screens/DetailsScreen.tsx
  • code/addons/review/src/screens/SummaryScreen.stories.tsx
  • code/addons/review/src/screens/SummaryScreen.tsx
  • code/core/src/core-server/change-detection/StoryDependencyGraphService.ts
  • code/core/src/core-server/change-detection/index.ts
  • code/core/src/core-server/change-detection/source-changes.test.ts
  • code/core/src/core-server/change-detection/source-changes.ts
  • code/core/src/core-server/index.ts
🚧 Files skipped from review as they are similar to previous changes (13)
  • code/addons/review/src/review-state.ts
  • code/core/src/core-server/change-detection/index.ts
  • code/addons/review/src/screens/DetailsScreen.stories.tsx
  • code/addons/review/src/constants.ts
  • code/core/src/core-server/index.ts
  • code/addons/review/src/screens/SummaryScreen.stories.tsx
  • code/addons/review/src/components/StaleBanner.tsx
  • code/core/src/core-server/change-detection/source-changes.test.ts
  • code/addons/review/src/preset.test.ts
  • code/addons/review/src/preset.ts
  • code/core/src/core-server/change-detection/source-changes.ts
  • code/addons/review/src/screens/DetailsScreen.tsx
  • code/addons/review/src/ReviewPage.ts

Update DetailsScreen/SummaryScreen Stale stories to assert the new banner
wording, and drop the SummaryScreen Minimal assertion for branchName, which
is no longer rendered in the summary header.
…ion' into stale-review-banner

# Conflicts:
#	code/addons/review/src/ReviewPage.ts
#	code/addons/review/src/screens/DetailsScreen.tsx
…ion' into stale-review-banner

# Conflicts:
#	code/addons/review/src/review-state.ts
#	code/addons/review/src/screens/DetailsScreen.tsx
#	code/addons/review/src/screens/SummaryScreen.tsx
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:normal Run our default set of CI jobs (choose this for most PRs). feature request qa:needed Pull Requests that will need manual QA prior to release.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant