Skip to content

feat: better, faster, smarter capture list #675

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
Jul 6, 2025
Merged

feat: better, faster, smarter capture list #675

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
8bf929f
feat: revamp selector generator with xpath logic
RohitR311 Jul 6, 2025
315cf94
feat: add xpath highlighting logic
RohitR311 Jul 6, 2025
33d9522
feat: add child selector caching, revamp dom highlighting
RohitR311 Jul 6, 2025
369b56f
feat: extract data via xpath logic
RohitR311 Jul 6, 2025
5f1084c
feat: revamp client list extractor with xpath logic
RohitR311 Jul 6, 2025
fec6e81
feat: rm network monitoring and caching
RohitR311 Jul 6, 2025
afe52a1
fix: list selection translation
RohitR311 Jul 6, 2025
82152c5
feat: add dom mode check
RohitR311 Jul 6, 2025
a677cb9
feat: add update dom mode
RohitR311 Jul 6, 2025
be42c1d
feat: update register editor events
RohitR311 Jul 6, 2025
9a065a3
feat: xpath support for core extraction
RohitR311 Jul 6, 2025
330997f
feat: cleanup on discard and confirm
RohitR311 Jul 6, 2025
f21ad1e
feat: add dom update methods
RohitR311 Jul 6, 2025
bd00405
Merge branch 'develop' into smart-list
RohitR311 Jul 6, 2025
1517f26
Merge branch 'develop' into smart-list
RohitR311 Jul 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,058 changes: 688 additions & 370 deletions maxun-core/src/browserSide/scraper.js
View file
Open in desktop

Large diffs are not rendered by default.

364 changes: 211 additions & 153 deletions server/src/browser-management/classes/RemoteBrowser.ts
View file
Open in desktop

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion server/src/workflow-management/classes/Generator.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -464,7 +464,6 @@ export class WorkflowGenerator {
public onClick = async (coordinates: Coordinates, page: Page) => {
let where: WhereWhatPair["where"] = { url: this.getBestUrl(page.url()) };
const selector = await this.generateSelector(page, coordinates, ActionType.Click);
console.log("COOORDINATES: ", coordinates);
logger.log('debug', `Element's selector: ${selector}`);

const elementInfo = await getElementInformation(page, coordinates, '', false);
Expand Down Expand Up @@ -999,6 +998,7 @@ export class WorkflowGenerator {
rect,
selector: displaySelector,
elementInfo,
isDOMMode: this.isDOMMode,
// Include shadow DOM specific information
shadowInfo: elementInfo?.isShadowRoot ? {
mode: elementInfo.shadowRootMode,
Expand Down
902 changes: 509 additions & 393 deletions src/components/browser/BrowserWindow.tsx
View file
Open in desktop

Large diffs are not rendered by default.

160 changes: 103 additions & 57 deletions src/components/recorder/DOMBrowserRenderer.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ interface RRWebDOMBrowserRendererProps {
getList?: boolean;
getText?: boolean;
listSelector?: string | null;
cachedChildSelectors?: string[];
paginationMode?: boolean;
paginationType?: string;
limitMode?: boolean;
Expand All @@ -106,12 +107,14 @@ interface RRWebDOMBrowserRendererProps {
selector: string;
elementInfo: ElementInfo | null;
childSelectors?: string[];
groupInfo?: any;
Copy link

@coderabbitai coderabbitai bot Jul 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

⚠️ Potential issue

Improve type safety by defining proper types for groupInfo.

The groupInfo parameter uses any type, which reduces type safety and makes the code harder to maintain.

Define a proper interface for groupInfo:

+interface GroupInfo {
+  isGroupElement: boolean;
+  groupSize: number;
+  groupElements: HTMLElement[];
+  groupFingerprint?: ElementFingerprint;
+}

 interface RRWebDOMBrowserRendererProps {
   // ... other props
   onHighlight?: (data: {
     rect: DOMRect;
     selector: string;
     elementInfo: ElementInfo | null;
     childSelectors?: string[];
-    groupInfo?: any;
+    groupInfo?: GroupInfo;
   }) => void;
   onElementSelect?: (data: {
     rect: DOMRect;
     selector: string;
     elementInfo: ElementInfo | null;
     childSelectors?: string[];
-    groupInfo?: any;
+    groupInfo?: GroupInfo;
   }) => void;

Also applies to: 117-117

🤖 Prompt for AI Agents 
In src/components/recorder/DOMBrowserRenderer.tsx at lines 110 and 117, the
groupInfo parameter is typed as any, which reduces type safety. Define a proper
TypeScript interface describing the expected structure of groupInfo and replace
the any type with this interface to improve maintainability and type safety.

}) => void;
onElementSelect?: (data: {
rect: DOMRect;
selector: string;
elementInfo: ElementInfo | null;
childSelectors?: string[];
groupInfo?: any;
}) => void;
onShowDatePicker?: (info: {
coordinates: { x: number; y: number };
Expand Down Expand Up @@ -144,6 +147,7 @@ export const DOMBrowserRenderer: React.FC<RRWebDOMBrowserRendererProps> = ({
getList = false,
getText = false,
listSelector = null,
cachedChildSelectors = [],
paginationMode = false,
paginationType = "",
limitMode = false,
Expand Down Expand Up @@ -205,11 +209,24 @@ export const DOMBrowserRenderer: React.FC<RRWebDOMBrowserRendererProps> = ({
const handleDOMHighlighting = useCallback(
(x: number, y: number, iframeDoc: Document) => {
try {
if (!getText && !getList) {
setCurrentHighlight(null);
if (onHighlight) {
onHighlight({
rect: new DOMRect(0, 0, 0, 0),
selector: "",
elementInfo: null,
});
}
return;
}

const highlighterData =
clientSelectorGenerator.generateDataForHighlighter(
{ x, y },
iframeDoc,
true
true,
cachedChildSelectors
);

if (!highlighterData) {
Expand All @@ -224,70 +241,40 @@ export const DOMBrowserRenderer: React.FC<RRWebDOMBrowserRendererProps> = ({
return;
}

const { rect, selector, elementInfo, childSelectors } = highlighterData;
const { rect, selector, elementInfo, childSelectors, groupInfo } =
highlighterData;

let shouldHighlight = false;

if (getList) {
if (listSelector) {
const hasValidChildSelectors =
Array.isArray(childSelectors) && childSelectors.length > 0;

// First phase: Allow any group to be highlighted for selection
if (!listSelector && groupInfo?.isGroupElement) {
shouldHighlight = true;
}
// Second phase: Show valid children within selected group
else if (listSelector) {
if (limitMode) {
shouldHighlight = false;
} else if (paginationMode) {
if (
paginationType !== "" &&
!["none", "scrollDown", "scrollUp"].includes(paginationType)
) {
shouldHighlight = true;
} else {
shouldHighlight = false;
}
} else if (childSelectors && childSelectors.includes(selector)) {
} else if (
paginationMode &&
paginationType !== "" &&
!["none", "scrollDown", "scrollUp"].includes(paginationType)
) {
shouldHighlight = true;
} else if (childSelectors && childSelectors.length > 0) {
console.log("✅ Child selectors present, highlighting enabled");
shouldHighlight = true;
} else if (elementInfo?.isIframeContent && childSelectors) {
const isIframeChild = childSelectors.some(
(childSelector: string) =>
selector.includes(":>>") &&
childSelector
.split(":>>")
.some((part) => selector.includes(part.trim()))
);
shouldHighlight = isIframeChild;
} else if (selector.includes(":>>") && hasValidChildSelectors) {
const selectorParts = selector
.split(":>>")
.map((part: string) => part.trim());
const isValidMixedSelector = selectorParts.some((part: any) =>
childSelectors!.some((childSelector) =>
childSelector.includes(part)
)
);
} else if (elementInfo?.isShadowRoot && childSelectors) {
const isShadowChild = childSelectors.some(
(childSelector: string) =>
selector.includes(">>") &&
childSelector
.split(">>")
.some((part) => selector.includes(part.trim()))
);
} else if (selector.includes(">>") && hasValidChildSelectors) {
const selectorParts = selector
.split(">>")
.map((part: string) => part.trim());
const isValidMixedSelector = selectorParts.some((part: any) =>
childSelectors!.some((childSelector) =>
childSelector.includes(part)
)
);
} else {
console.log("❌ No child selectors available");
shouldHighlight = false;
}
} else {
}
// No list selector - show regular highlighting
else {
shouldHighlight = true;
}
} else {
// getText mode - always highlight
shouldHighlight = true;
}

Expand Down Expand Up @@ -316,6 +303,7 @@ export const DOMBrowserRenderer: React.FC<RRWebDOMBrowserRendererProps> = ({
},
selector,
childSelectors,
groupInfo,
});
}
}
Expand All @@ -335,9 +323,11 @@ export const DOMBrowserRenderer: React.FC<RRWebDOMBrowserRendererProps> = ({
}
},
[
getText,
getList,
listSelector,
paginationMode,
cachedChildSelectors,
paginationType,
limitMode,
onHighlight,
Expand All @@ -363,6 +353,10 @@ export const DOMBrowserRenderer: React.FC<RRWebDOMBrowserRendererProps> = ({
return;
}

if (!isInCaptureMode) {
return;
}

const now = performance.now();
if (now - lastMouseMoveTime.current < MOUSE_MOVE_THROTTLE) {
return;
Expand Down Expand Up @@ -401,11 +395,24 @@ export const DOMBrowserRenderer: React.FC<RRWebDOMBrowserRendererProps> = ({
e.stopPropagation();

if (currentHighlight && onElementSelect) {
// Get the group info for the current highlight
const highlighterData =
clientSelectorGenerator.generateDataForHighlighter(
{ x: iframeX, y: iframeY },
iframeDoc,
true,
cachedChildSelectors
);

onElementSelect({
rect: currentHighlight.rect,
selector: currentHighlight.selector,
elementInfo: currentHighlight.elementInfo,
childSelectors: currentHighlight.childSelectors || [],
childSelectors:
cachedChildSelectors.length > 0
? cachedChildSelectors
: highlighterData?.childSelectors || [],
groupInfo: highlighterData?.groupInfo,
});
Comment on lines +398 to 416
Copy link

@coderabbitai coderabbitai bot Jul 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

⚠️ Potential issue

Remove redundant selector generation call.

The code retrieves highlighter data again to get groupInfo, but this information should already be available from the currentHighlight state that was set during the last mouse move.

Use the existing highlight data:

 if (currentHighlight && onElementSelect) {
-  // Get the group info for the current highlight
-  const highlighterData =
-    clientSelectorGenerator.generateDataForHighlighter(
-      { x: iframeX, y: iframeY },
-      iframeDoc,
-      true,
-      cachedChildSelectors
-    );
-
   onElementSelect({
     rect: currentHighlight.rect,
     selector: currentHighlight.selector,
     elementInfo: currentHighlight.elementInfo,
     childSelectors:
       cachedChildSelectors.length > 0
         ? cachedChildSelectors
-        : highlighterData?.childSelectors || [],
-    groupInfo: highlighterData?.groupInfo,
+        : currentHighlight.childSelectors || [],
+    groupInfo: currentHighlight.groupInfo,
   });
 }

To support this, update the currentHighlight state interface to include groupInfo.

🤖 Prompt for AI Agents 
In src/components/recorder/DOMBrowserRenderer.tsx around lines 398 to 416,
remove the redundant call to clientSelectorGenerator.generateDataForHighlighter
since groupInfo is already present in currentHighlight. Update the
currentHighlight state interface to include groupInfo if it does not already,
then use currentHighlight.groupInfo directly in the onElementSelect call instead
of calling generateDataForHighlighter again.

}
notifyLastAction("select element");
Expand Down Expand Up @@ -790,12 +797,41 @@ export const DOMBrowserRenderer: React.FC<RRWebDOMBrowserRendererProps> = ({

rebuiltHTML = "<!DOCTYPE html>\n" + rebuiltHTML;

const additionalCSS = [];

if (snapshotData.resources.fonts?.length > 0) {
const fontCSS = snapshotData.resources.fonts
.map((font) => {
const format = font.format || "woff2";
return `
@font-face {
font-family: 'ProxiedFont-${
font.url.split("/").pop()?.split(".")[0] || "unknown"
}';
src: url("${font.dataUrl}") format("${format}");
font-display: swap;
}
`;
})
.join("\n");
additionalCSS.push(fontCSS);
}

if (snapshotData.resources.stylesheets?.length > 0) {
const externalCSS = snapshotData.resources.stylesheets
.map((stylesheet) => stylesheet.content)
.join("\n\n");
additionalCSS.push(externalCSS);
}

const enhancedCSS = `
/* rrweb rebuilt content styles */
html, body {
margin: 0 !important;
padding: 8px !important;
overflow-x: hidden !important;
margin: 0 !important;
padding: 8px !important;
font-family: system-ui, -apple-system, BlinkMacSystemFont, sans-serif !important;
background: white !important;
overflow-x: hidden !important;
}

html::-webkit-scrollbar,
Expand All @@ -818,12 +854,22 @@ export const DOMBrowserRenderer: React.FC<RRWebDOMBrowserRendererProps> = ({
scrollbar-width: none !important; /* Firefox */
-ms-overflow-style: none !important; /* Internet Explorer 10+ */
}

img {
max-width: 100% !important;
height: auto !important;
}


/* Make everything interactive */
* {
cursor: "pointer" !important;
}
`;

/* Additional CSS from resources */
${additionalCSS.join("\n\n")}
`;


const headTagRegex = /<head[^>]*>/i;
const cssInjection = `
Expand Down
Loading