Skip to content

Conversation

@RohitR311
Copy link
Contributor

@RohitR311 RohitR311 commented Jun 25, 2025

What this PR does?

  1. Displays screenshot output preview data almost instantly
  2. Fixes the issue of navigation failing after triggering output preview

Closes: #658
Fixes: #657

Summary by CodeRabbit

Summary by CodeRabbit

  • New Features

    • Added support for storing and updating additional screenshot-related data for individual steps.
    • Enabled direct screenshot capture and automatic updating of screenshot data via real-time events.
    • Improved preview panel control with a new toggle for displaying captured data previews.
  • Chores

    • Refactored interpretation log to consolidate data updates and remove socket event handling.

@RohitR311 RohitR311 added Type: Bug Something isn't working Type: Enhancement Improvements to existing features Scope: Recorder All issues/PRs related to recorder labels Jun 25, 2025
@coderabbitai
Copy link

coderabbitai bot commented Jun 25, 2025

"""

Walkthrough

A new optional property screenshotData has been added to the ScreenshotStep interface, and a context method updateScreenshotStepData is introduced to update this property for specific screenshot steps. The method is incorporated into the context provider, and redundant comments on actionId properties are removed. The RightSidePanel component now listens for a directScreenshotCaptured socket event to update the screenshot step data and emits a captureDirectScreenshot event when capturing screenshots. The InterpretationLog component is refactored to remove socket event handling and instead updates capture data from the browserSteps context, adding a showPreviewData state to control preview visibility.

Changes

File(s) Change Summary
src/context/browserSteps.tsx Added screenshotData to ScreenshotStep, implemented updateScreenshotStepData context method, cleaned up comments.
src/components/recorder/RightSidePanel.tsx Added socket event listener for directScreenshotCaptured to update screenshot step data, emit captureDirectScreenshot event in screenshot capture flow.
src/components/run/InterpretationLog.tsx Removed socket event handling and related states, updated capture data from browserSteps context, added showPreviewData state for preview control.
server/src/browser-management/classes/RemoteBrowser.ts Added captureDirectScreenshot method and socket event handlers for direct screenshot capture requests.
src/components/recorder/DOMBrowserRenderer.tsx Changed CSS cursor style to always "pointer !important" inside rrweb snapshot and overlay.
src/components/recorder/SidePanelHeader.tsx Added optional onPreviewClick callback prop to SidePanelHeader component.
src/components/run/InterpretationButtons.tsx Added optional onPreviewComplete callback prop; simplified handlePlay to call this callback immediately.

Sequence Diagram(s)

sequenceDiagram
    participant RightSidePanel
    participant Socket
    participant BrowserStepsProvider
    participant RemoteBrowser

    RightSidePanel->>Socket: emit captureDirectScreenshot(screenshotSettings)
    Socket->>RemoteBrowser: captureDirectScreenshot(settings)
    RemoteBrowser->>Socket: emit directScreenshotCaptured(screenshotData)
    Socket->>RightSidePanel: directScreenshotCaptured(screenshotData)
    RightSidePanel->>BrowserStepsProvider: updateScreenshotStepData(id, screenshotData)
    BrowserStepsProvider->>BrowserStepsProvider: update step with matching id
Loading

Assessment against linked issues

Objective Addressed Explanation
Fix navigation failure after output preview (#657) Changes improve screenshot handling and preview display but do not explicitly address navigation or snapshot refresh after preview.

Assessment against linked issues: Out-of-scope changes

Code Change Explanation
Addition of screenshotData property and updateScreenshotStepData method (src/context/browserSteps.tsx) These changes introduce screenshot metadata management, unrelated to navigation or output preview bug.
Socket event handling for direct screenshot capture (src/components/recorder/RightSidePanel.tsx) This feature relates to screenshot capture flow, not to navigation or output preview functionality.
Refactor of InterpretationLog to remove socket handling and update from context (src/components/run/InterpretationLog.tsx) Refactoring for data management and preview control unrelated to navigation or output preview bug.
Addition of captureDirectScreenshot method and socket handlers (server/src/browser-management/classes/RemoteBrowser.ts) This backend feature supports direct screenshot capture, unrelated to navigation or output preview bug.
Cursor style changes in rrweb snapshot rendering (src/components/recorder/DOMBrowserRenderer.tsx) Cosmetic UI change unrelated to navigation or output preview bug.
Addition of onPreviewClick prop to SidePanelHeader and onPreviewComplete prop to InterpretationButtons UI callback enhancements unrelated to navigation or output preview bug.

Possibly related PRs

  • feat: faster output preview #583: Adds methods to update list step data in the browser steps context, similar to this PR’s update of screenshot step data, both enhancing step data management.

Poem

🐇 In code's quiet burrow, new data takes flight,
Screenshots enriched with a fresh, glowing light.
From sockets they travel, to context they flow,
Updating each step so the previews can show.
A hop, a click, the preview's alive—
With every new change, our code will thrive! 🌟
📸✨
"""

✨ Finishing Touches
  • 📝 Generate Docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 09064e9 and 1a6dc32.

📒 Files selected for processing (1)
  • src/components/recorder/RightSidePanel.tsx (3 hunks)
🔇 Additional comments (2)
src/components/recorder/RightSidePanel.tsx (2)

75-75: LGTM! Clean addition of new context method.

The updateScreenshotStepData method is properly added to the destructuring from useBrowserSteps, following the existing pattern and naming conventions.


675-690: Excellent type safety improvements and direct screenshot integration.

The changes enhance the function in two important ways:

  1. Type safety: The as const assertions prevent TypeScript from widening the literal string types, ensuring better type checking.
  2. Direct screenshot capture: The new captureDirectScreenshot event emission enables the faster screenshot preview functionality described in the PR objectives.

The order of socket emissions is correct - triggering the direct capture first, then the standard action flow.

Comment on lines +186 to +208
useEffect(() => {
if (socket) {
const handleDirectScreenshot = (data: any) => {
const screenshotSteps = browserSteps.filter(step =>
step.type === 'screenshot' && step.actionId === currentScreenshotActionId
);

if (screenshotSteps.length > 0) {
const latestStep = screenshotSteps[screenshotSteps.length - 1];
updateScreenshotStepData(latestStep.id, data.screenshot);
}

setCurrentScreenshotActionId('');
};

socket.on('directScreenshotCaptured', handleDirectScreenshot);

return () => {
socket.off('directScreenshotCaptured', handleDirectScreenshot);
};
}
}, [socket, id, notify, t, currentScreenshotActionId, updateScreenshotStepData, setCurrentScreenshotActionId]);

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Optimize dependencies and add error handling.

The socket event handling logic is correct, but consider these improvements:

  1. Remove unused dependencies - id, notify, and t are in the dependency array but not used in the effect, causing unnecessary re-renders.
  2. Add error handling for potentially malformed socket data.

Apply this diff to optimize the effect:

  useEffect(() => {
    if (socket) {
      const handleDirectScreenshot = (data: any) => {
+       try {
+         if (!data?.screenshot) {
+           console.warn('Invalid screenshot data received:', data);
+           return;
+         }
+
          const screenshotSteps = browserSteps.filter(step => 
            step.type === 'screenshot' && step.actionId === currentScreenshotActionId
          );
          
          if (screenshotSteps.length > 0) {
            const latestStep = screenshotSteps[screenshotSteps.length - 1];          
            updateScreenshotStepData(latestStep.id, data.screenshot);
          }
          
          setCurrentScreenshotActionId('');
+       } catch (error) {
+         console.error('Error handling direct screenshot:', error);
+       }
      };

      socket.on('directScreenshotCaptured', handleDirectScreenshot);

      return () => {
        socket.off('directScreenshotCaptured', handleDirectScreenshot);
      };
    }
-  }, [socket, id, notify, t, currentScreenshotActionId, updateScreenshotStepData, setCurrentScreenshotActionId]);
+  }, [socket, currentScreenshotActionId, updateScreenshotStepData, setCurrentScreenshotActionId, browserSteps]);
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
useEffect(() => {
if (socket) {
const handleDirectScreenshot = (data: any) => {
const screenshotSteps = browserSteps.filter(step =>
step.type === 'screenshot' && step.actionId === currentScreenshotActionId
);
if (screenshotSteps.length > 0) {
const latestStep = screenshotSteps[screenshotSteps.length - 1];
updateScreenshotStepData(latestStep.id, data.screenshot);
}
setCurrentScreenshotActionId('');
};
socket.on('directScreenshotCaptured', handleDirectScreenshot);
return () => {
socket.off('directScreenshotCaptured', handleDirectScreenshot);
};
}
}, [socket, id, notify, t, currentScreenshotActionId, updateScreenshotStepData, setCurrentScreenshotActionId]);
useEffect(() => {
if (socket) {
const handleDirectScreenshot = (data: any) => {
try {
if (!data?.screenshot) {
console.warn('Invalid screenshot data received:', data);
return;
}
const screenshotSteps = browserSteps.filter(step =>
step.type === 'screenshot' && step.actionId === currentScreenshotActionId
);
if (screenshotSteps.length > 0) {
const latestStep = screenshotSteps[screenshotSteps.length - 1];
updateScreenshotStepData(latestStep.id, data.screenshot);
}
setCurrentScreenshotActionId('');
} catch (error) {
console.error('Error handling direct screenshot:', error);
}
};
socket.on('directScreenshotCaptured', handleDirectScreenshot);
return () => {
socket.off('directScreenshotCaptured', handleDirectScreenshot);
};
}
}, [
socket,
currentScreenshotActionId,
updateScreenshotStepData,
setCurrentScreenshotActionId,
browserSteps,
]);
🤖 Prompt for AI Agents
In src/components/recorder/RightSidePanel.tsx around lines 186 to 208, remove
the unused dependencies 'id', 'notify', and 't' from the useEffect dependency
array to prevent unnecessary re-renders. Additionally, add error handling inside
the 'handleDirectScreenshot' function to safely handle cases where the socket
data might be malformed, such as checking if 'data' and 'data.screenshot' exist
before calling 'updateScreenshotStepData'.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🔭 Outside diff range comments (1)
src/components/run/InterpretationLog.tsx (1)

59-69: Missing dependency in updateActiveTab callback.

The callback uses getAvailableTabs() but doesn't include it in the dependency array, which could lead to stale closure issues.

Apply this diff to fix the missing dependency:

-  }, [captureListData.length, captureTextData.length, screenshotData.length]);
+  }, [captureListData.length, captureTextData.length, screenshotData.length, getAvailableTabs]);
🧹 Nitpick comments (3)
src/components/run/InterpretationLog.tsx (3)

88-90: Simplify type assertion for better readability.

The inline type assertion is verbose. Consider defining a proper type or using type guards.

Consider defining a type at the component level:

+type ScreenshotStep = {
+  type: 'screenshot';
+  id: number;
+  fullPage: boolean;
+  actionId?: string;
+  screenshotData?: string;
+};

    const screenshotSteps = browserSteps.filter(step => 
      step.type === 'screenshot'
-    ) as Array<{ type: 'screenshot'; id: number; fullPage: boolean; actionId?: string; screenshotData?: string }>;
+    ) as ScreenshotStep[];

92-96: Consider avoiding non-null assertion for clarity.

While the non-null assertion is safe due to the filter, it might be clearer to avoid it.

-      const screenshots = screenshotsWithData.map(step => step.screenshotData!);
+      const screenshots = screenshotsWithData.map(step => step.screenshotData).filter(Boolean) as string[];

71-99: Consider memoizing browser steps processing for performance.

The effect processes all browser steps on every change, which could impact performance with large datasets.

Consider using useMemo to memoize the processed data:

const { textData, listData, screenshotData } = useMemo(() => {
  const textSteps = browserSteps.filter(step => step.type === 'text');
  const textDataRow = textSteps.length > 0 
    ? [Object.fromEntries(textSteps.map(step => [step.label, step.data]))]
    : [];
    
  const listSteps = browserSteps.filter(step => step.type === 'list');
  
  const screenshotSteps = browserSteps.filter(step => 
    step.type === 'screenshot' && step.screenshotData
  );
  const screenshots = screenshotSteps.map(step => step.screenshotData!);
  
  return {
    textData: textDataRow,
    listData: listSteps,
    screenshotData: screenshots
  };
}, [browserSteps]);

useEffect(() => {
  setCaptureTextData(textData);
  setCaptureListData(listData);
  setScreenshotData(screenshotData);
  updateActiveTab();
}, [textData, listData, screenshotData, updateActiveTab]);
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 1a6dc32 and 6768dda.

📒 Files selected for processing (1)
  • src/components/run/InterpretationLog.tsx (5 hunks)
🔇 Additional comments (2)
src/components/run/InterpretationLog.tsx (2)

43-46: Good addition of preview visibility control state.

The new showPreviewData state provides clear control over when preview data should be displayed, aligning with the PR objective to improve the screenshot output preview feature.


436-436: Clean implementation of preview trigger.

The onPreviewClick callback properly enables the preview display, supporting the PR objective of instant screenshot output preview.


return tabs;
}, [captureListData.length, captureTextData.length, screenshotData.length]);
}, [captureListData.length, captureTextData.length, screenshotData.length, showPreviewData]);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Remove unused dependency from getAvailableTabs callback.

The showPreviewData variable is not used within the callback function, so it shouldn't be in the dependency array.

-  }, [captureListData.length, captureTextData.length, screenshotData.length, showPreviewData]);
+  }, [captureListData.length, captureTextData.length, screenshotData.length]);
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
}, [captureListData.length, captureTextData.length, screenshotData.length, showPreviewData]);
}, [captureListData.length, captureTextData.length, screenshotData.length]);
🤖 Prompt for AI Agents
In src/components/run/InterpretationLog.tsx at line 129, the dependency array
for the getAvailableTabs callback includes showPreviewData, which is not used
inside the callback. Remove showPreviewData from the dependency array to avoid
unnecessary re-renders and keep dependencies accurate.

@RohitR311 RohitR311 requested a review from amhsirak June 25, 2025 07:45
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🧹 Nitpick comments (2)
server/src/browser-management/classes/RemoteBrowser.ts (1)

1897-1909: Consider memory optimization for screenshot data.

Converting large screenshots to base64 can be memory-intensive. Consider implementing streaming or chunking for better memory management.

 const screenshotBuffer = await this.currentPage.screenshot({
   fullPage: settings.fullPage,
   type: settings.type || 'png',
   timeout: settings.timeout || 30000,
   animations: settings.animations || 'allow',
   caret: settings.caret || 'hide',
   scale: settings.scale || 'device'
 });

-const base64Data = screenshotBuffer.toString('base64');
-const mimeType = `image/${settings.type || 'png'}`;
-const dataUrl = `data:${mimeType};base64,${base64Data}`;
+// Optimize memory usage by processing screenshot buffer
+const mimeType = `image/${settings.type || 'png'}`;
+let dataUrl: string;
+try {
+  const base64Data = screenshotBuffer.toString('base64');
+  dataUrl = `data:${mimeType};base64,${base64Data}`;
+} finally {
+  // Help GC by nullifying the buffer reference
+  (screenshotBuffer as any) = null;
+}
src/components/recorder/DOMBrowserRenderer.tsx (1)

1130-1131: Cursor styling may be ineffective due to disabled pointer events.

The cursor style on this overlay may not be visible to users since pointerEvents: "none" is set. If cursor styling is needed, consider using "crosshair" to match capture mode expectations.

-            cursor: "pointer !important",
+            cursor: "crosshair !important",

Or remove the cursor styling entirely since pointer events are disabled.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 6768dda and 390c5f2.

📒 Files selected for processing (6)
  • server/src/browser-management/classes/RemoteBrowser.ts (2 hunks)
  • src/components/recorder/DOMBrowserRenderer.tsx (2 hunks)
  • src/components/recorder/RightSidePanel.tsx (3 hunks)
  • src/components/recorder/SidePanelHeader.tsx (2 hunks)
  • src/components/run/InterpretationButtons.tsx (3 hunks)
  • src/context/browserSteps.tsx (5 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
  • src/context/browserSteps.tsx
  • src/components/recorder/RightSidePanel.tsx
🧰 Additional context used
🧬 Code Graph Analysis (1)
src/components/recorder/SidePanelHeader.tsx (1)
src/components/run/InterpretationButtons.tsx (1)
  • InterpretationButtons (26-190)
🔇 Additional comments (3)
server/src/browser-management/classes/RemoteBrowser.ts (1)

1869-1924: LGTM! Well-implemented direct screenshot capture method.

The method correctly implements direct screenshot capture with proper error handling and socket event emission. The implementation enables faster screenshot preview as intended by the PR objectives.

src/components/recorder/SidePanelHeader.tsx (1)

5-7: LGTM! Clean interface design.

The optional callback prop is well-defined and follows React conventions.

src/components/run/InterpretationButtons.tsx (1)

13-13: LGTM! Proper optional callback prop.

The callback prop is correctly typed as optional.

Comment on lines +1935 to +1943
this.socket.on(`captureDirectScreenshot:${this.userId}`, async (settings) => {
logger.debug(`Direct screenshot capture requested for user ${this.userId}`);
await this.captureDirectScreenshot(settings);
});

// For backward compatibility
this.socket.on('captureDirectScreenshot', async (settings) => {
await this.captureDirectScreenshot(settings);
});
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Add error handling and rate limiting for screenshot events.

The event handlers lack explicit error handling and rate limiting, which could lead to resource abuse or unhandled errors.

+// Add rate limiting for screenshot captures
+private lastScreenshotTime = 0;
+private readonly SCREENSHOT_RATE_LIMIT = 1000; // 1 second

 this.socket.on(`captureDirectScreenshot:${this.userId}`, async (settings) => {
-  logger.debug(`Direct screenshot capture requested for user ${this.userId}`);
-  await this.captureDirectScreenshot(settings);
+  try {
+    const now = Date.now();
+    if (now - this.lastScreenshotTime < this.SCREENSHOT_RATE_LIMIT) {
+      logger.warn(`Screenshot rate limit exceeded for user ${this.userId}`);
+      this.socket.emit('screenshotError', {
+        userId: this.userId,
+        error: 'Rate limit exceeded. Please wait before requesting another screenshot.'
+      });
+      return;
+    }
+    this.lastScreenshotTime = now;
+    
+    logger.debug(`Direct screenshot capture requested for user ${this.userId}`);
+    await this.captureDirectScreenshot(settings);
+  } catch (error) {
+    logger.error(`Error handling screenshot request for user ${this.userId}:`, error);
+    this.socket.emit('screenshotError', {
+      userId: this.userId,
+      error: 'Failed to process screenshot request'
+    });
+  }
 });

 // For backward compatibility
 this.socket.on('captureDirectScreenshot', async (settings) => {
-  await this.captureDirectScreenshot(settings);
+  try {
+    const now = Date.now();
+    if (now - this.lastScreenshotTime < this.SCREENSHOT_RATE_LIMIT) {
+      logger.warn(`Screenshot rate limit exceeded`);
+      this.socket.emit('screenshotError', {
+        userId: this.userId,
+        error: 'Rate limit exceeded. Please wait before requesting another screenshot.'
+      });
+      return;
+    }
+    this.lastScreenshotTime = now;
+    
+    await this.captureDirectScreenshot(settings);
+  } catch (error) {
+    logger.error(`Error handling backward compatibility screenshot request:`, error);
+    this.socket.emit('screenshotError', {
+      userId: this.userId,
+      error: 'Failed to process screenshot request'
+    });
+  }
 });

Committable suggestion skipped: line range outside the PR's diff.

🤖 Prompt for AI Agents
In server/src/browser-management/classes/RemoteBrowser.ts around lines 1935 to
1943, the event handlers for 'captureDirectScreenshot' lack error handling and
rate limiting. Wrap the handler logic in try-catch blocks to catch and log any
errors during screenshot capture. Implement rate limiting to restrict how
frequently a user can trigger these events, preventing resource abuse. Ensure
both the user-specific and backward-compatible event handlers include these
protections.

Comment on lines +1874 to +1881
public captureDirectScreenshot = async (settings: {
fullPage: boolean;
type: 'png' | 'jpeg';
timeout?: number;
animations?: 'disabled' | 'allow';
caret?: 'hide' | 'initial';
scale?: 'css' | 'device';
}): Promise<void> => {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Add input validation for screenshot settings.

The method accepts settings from the client without validation. Malicious or invalid values could cause issues.

 public captureDirectScreenshot = async (settings: {
   fullPage: boolean;
   type: 'png' | 'jpeg';
   timeout?: number;
   animations?: 'disabled' | 'allow';
   caret?: 'hide' | 'initial';
   scale?: 'css' | 'device';
 }): Promise<void> => {
+  // Validate settings
+  if (settings.timeout && (settings.timeout < 0 || settings.timeout > 60000)) {
+    logger.error("Invalid timeout value");
+    this.socket.emit('screenshotError', {
+      userId: this.userId,
+      error: 'Invalid timeout value (must be 0-60000ms)'
+    });
+    return;
+  }
+
+  if (!['png', 'jpeg'].includes(settings.type)) {
+    logger.error("Invalid image type");
+    this.socket.emit('screenshotError', {
+      userId: this.userId,
+      error: 'Invalid image type (must be png or jpeg)'
+    });
+    return;
+  }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
public captureDirectScreenshot = async (settings: {
fullPage: boolean;
type: 'png' | 'jpeg';
timeout?: number;
animations?: 'disabled' | 'allow';
caret?: 'hide' | 'initial';
scale?: 'css' | 'device';
}): Promise<void> => {
public captureDirectScreenshot = async (settings: {
fullPage: boolean;
type: 'png' | 'jpeg';
timeout?: number;
animations?: 'disabled' | 'allow';
caret?: 'hide' | 'initial';
scale?: 'css' | 'device';
}): Promise<void> => {
// Validate settings
if (settings.timeout && (settings.timeout < 0 || settings.timeout > 60000)) {
logger.error("Invalid timeout value");
this.socket.emit('screenshotError', {
userId: this.userId,
error: 'Invalid timeout value (must be 0-60000ms)'
});
return;
}
if (!['png', 'jpeg'].includes(settings.type)) {
logger.error("Invalid image type");
this.socket.emit('screenshotError', {
userId: this.userId,
error: 'Invalid image type (must be png or jpeg)'
});
return;
}
// …rest of existing implementation…
🤖 Prompt for AI Agents
In server/src/browser-management/classes/RemoteBrowser.ts around lines 1874 to
1881, the captureDirectScreenshot method accepts settings without validation,
which risks processing invalid or malicious inputs. Add input validation to
check that each setting property (fullPage, type, timeout, animations, caret,
scale) is of the expected type and within allowed values before proceeding. If
any validation fails, throw an appropriate error or reject the input to prevent
further processing.

/* Make everything interactive */
* {
cursor: ${isInCaptureMode ? "crosshair" : "pointer"} !important;
cursor: "pointer" !important;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Consider restoring conditional cursor styling for better UX.

The hardcoded "pointer" cursor removes the visual distinction between normal navigation and capture mode. Users typically expect a crosshair cursor when selecting elements (isInCaptureMode = true) to clearly indicate the interaction mode.

Consider reverting to conditional cursor styling:

-              cursor: "pointer" !important; 
+              cursor: ${isInCaptureMode ? '"crosshair"' : '"pointer"'} !important;
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
cursor: "pointer" !important;
cursor: ${isInCaptureMode ? '"crosshair"' : '"pointer"'} !important;
🤖 Prompt for AI Agents
In src/components/recorder/DOMBrowserRenderer.tsx at line 859, the cursor style
is hardcoded to "pointer" which removes the visual distinction between normal
navigation and capture mode. Modify the cursor style to be conditional based on
the isInCaptureMode state, setting it to "crosshair" when isInCaptureMode is
true and "pointer" otherwise, to improve user experience by clearly indicating
the interaction mode.

Comment on lines +21 to +24
<InterpretationButtons
enableStepping={(isPaused) => setSteppingIsDisabled(!isPaused)}
onPreviewComplete={onPreviewClick}
/>
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Verify the prop name mapping consistency.

The prop onPreviewClick is mapped to onPreviewComplete when passed to InterpretationButtons. While this works functionally, the naming suggests different semantic meanings - "click" vs "complete". Consider aligning the naming for better clarity.


🏁 Script executed:

#!/bin/bash
# Description: Check if onPreviewComplete is consistently used across the codebase
# Expected: Find usages to verify naming consistency

rg -A 3 -B 3 "onPreviewComplete|onPreviewClick"

Length of output: 3638


Align prop naming between SidePanelHeader and InterpretationButtons

To avoid semantic confusion, the “preview” callback should use the same name in both components. Right now:

  • InterpretationButtons defines its callback prop as onPreviewComplete.
  • SidePanelHeader exposes its callback as onPreviewClick and then maps it to onPreviewComplete.

Please consolidate on one name—preferably onPreviewComplete since the callback fires when the preview is done.

Changes needed:

  • In src/components/recorder/SidePanelHeader.tsx:
    • Rename the prop in the interface and component signature from onPreviewClick?: () => void to onPreviewComplete?: () => void.
    • Update the JSX that passes it down:
      - <InterpretationButtons 
      -   enableStepping={(isPaused) => setSteppingIsDisabled(!isPaused)} 
      -   onPreviewComplete={onPreviewClick}
      - />
      + <InterpretationButtons 
      +   enableStepping={(isPaused) => setSteppingIsDisabled(!isPaused)} 
      +   onPreviewComplete={onPreviewComplete}
      + />
  • In src/components/run/InterpretationLog.tsx:
    • Update the usage of SidePanelHeader:
      - <SidePanelHeader onPreviewClick={() => setShowPreviewData(true)} />
      + <SidePanelHeader onPreviewComplete={() => setShowPreviewData(true)} />

With these updates, both components will share the same prop name and intent.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
<InterpretationButtons
enableStepping={(isPaused) => setSteppingIsDisabled(!isPaused)}
onPreviewComplete={onPreviewClick}
/>
<InterpretationButtons
enableStepping={(isPaused) => setSteppingIsDisabled(!isPaused)}
onPreviewComplete={onPreviewComplete}
/>
🤖 Prompt for AI Agents
In src/components/recorder/SidePanelHeader.tsx around lines 21 to 24, rename the
prop and interface member from onPreviewClick to onPreviewComplete to align with
InterpretationButtons. Update the component signature and all references
accordingly, including the JSX where the prop is passed down. Then, in
src/components/run/InterpretationLog.tsx, update the usage of SidePanelHeader to
use onPreviewComplete instead of onPreviewClick to maintain consistency across
components.

}
}
onPreviewComplete?.();
notify('info', t('interpretation_buttons.messages.run_finished'));
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Update notification message for accuracy.

The notification message references "run_finished" but the operation is now a preview. This could confuse users about what actually happened.

Consider adding a more appropriate translation key like preview_completed or preview_ready.

🤖 Prompt for AI Agents
In src/components/run/InterpretationButtons.tsx at line 107, the notification
message uses the translation key 'interpretation_buttons.messages.run_finished',
which inaccurately describes the operation as a run completion instead of a
preview. Update the translation key to a more appropriate one such as
'preview_completed' or 'preview_ready' to accurately reflect the preview
operation in the notification message.

Comment on lines 105 to +119
const handlePlay = async () => {
if (!info.running) {
setInfo({ ...info, running: true });
const finished = await interpretCurrentRecording();
setInfo({ ...info, running: false });
if (finished) {
notify('info', t('interpretation_buttons.messages.run_finished'));
} else {
notify('error', t('interpretation_buttons.messages.run_failed'));
}
}
onPreviewComplete?.();
notify('info', t('interpretation_buttons.messages.run_finished'));

// Legacy code for running the interpretation

// if (!info.running) {
// setInfo({ ...info, running: true });
// // const finished = await interpretCurrentRecording();
// setInfo({ ...info, running: false });
// if (finished) {
// } else {
// notify('error', t('interpretation_buttons.messages.run_failed'));
// }
// }
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Critical issues with the simplified handlePlay logic.

The refactored handlePlay function has several concerning issues:

  1. Misleading user feedback: The notification shows "run finished" immediately, but no actual processing occurs
  2. Broken state management: The info.running state is never set to true, so the button disable logic won't work properly during async operations
  3. Missing error handling: The callback execution has no try-catch protection
  4. Inconsistent behavior: The button text still shows "preview" but the notification suggests a "run" completed

Consider this improved implementation:

 const handlePlay = async () => {
+  if (info.running) return;
+  
+  try {
+    setInfo({ ...info, running: true });
     onPreviewComplete?.();
-    notify('info', t('interpretation_buttons.messages.run_finished'));
+    notify('info', t('interpretation_buttons.messages.preview_ready'));
+  } catch (error) {
+    notify('error', t('interpretation_buttons.messages.preview_failed'));
+  } finally {
+    setInfo({ ...info, running: false });
+  }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const handlePlay = async () => {
if (!info.running) {
setInfo({ ...info, running: true });
const finished = await interpretCurrentRecording();
setInfo({ ...info, running: false });
if (finished) {
notify('info', t('interpretation_buttons.messages.run_finished'));
} else {
notify('error', t('interpretation_buttons.messages.run_failed'));
}
}
onPreviewComplete?.();
notify('info', t('interpretation_buttons.messages.run_finished'));
// Legacy code for running the interpretation
// if (!info.running) {
// setInfo({ ...info, running: true });
// // const finished = await interpretCurrentRecording();
// setInfo({ ...info, running: false });
// if (finished) {
// } else {
// notify('error', t('interpretation_buttons.messages.run_failed'));
// }
// }
const handlePlay = async () => {
if (info.running) return;
try {
setInfo({ ...info, running: true });
onPreviewComplete?.();
notify('info', t('interpretation_buttons.messages.preview_ready'));
} catch (error) {
notify('error', t('interpretation_buttons.messages.preview_failed'));
} finally {
setInfo({ ...info, running: false });
}
// Legacy code for running the interpretation
// if (!info.running) {
// setInfo({ ...info, running: true });
// // const finished = await interpretCurrentRecording();
// setInfo({ ...info, running: false });
// if (finished) {
// } else {
// notify('error', t('interpretation_buttons.messages.run_failed'));
// }
// }
};
🤖 Prompt for AI Agents
In src/components/run/InterpretationButtons.tsx around lines 105 to 119, the
handlePlay function prematurely shows a "run finished" notification without
running any process, does not update the info.running state to true during
execution, lacks try-catch error handling, and causes inconsistent UI feedback.
To fix this, restore the logic to set info.running to true before starting the
async operation, wrap the async call in try-catch to handle errors and notify
accordingly, update info.running to false after completion, and ensure the
notification and button text accurately reflect the current state of the
operation.

@amhsirak amhsirak merged commit 3b7f017 into develop Jun 29, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Recorder All issues/PRs related to recorder Type: Bug Something isn't working Type: Enhancement Improvements to existing features

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feat: Faster screenshot output preview Bug: Navigation fails after triggering output preview

3 participants