Skip to content

feat: change credentials based on input type #375

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jan 23, 2025
Merged

feat: change credentials based on input type #375

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
1bc153d
feat: check for keyboard actions
RohitR311 Jan 22, 2025
2af89a0
feat: add type arg for keyboard actions
RohitR311 Jan 22, 2025
8419875
feat: add support for different inputs
RohitR311 Jan 22, 2025
ef3ec29
feat: add types for credentials
RohitR311 Jan 22, 2025
ee65ea8
feat: add input type for workflow type actions
RohitR311 Jan 22, 2025
1950cc5
feat: rm letter on backspace
RohitR311 Jan 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions maxun-core/src/interpret.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -490,6 +490,14 @@ export default class Interpreter extends EventEmitter {

const executeAction = async (invokee: any, methodName: string, args: any) => {
console.log("Executing action:", methodName, args);

if (methodName === 'press' || methodName === 'type') {
// Extract only the first two arguments for these methods
const limitedArgs = Array.isArray(args) ? args.slice(0, 2) : [args];
await (<any>invokee[methodName])(...limitedArgs);
return;
}

if (!args || Array.isArray(args)) {
await (<any>invokee[methodName])(...(args ?? []));
} else {
Expand Down
34 changes: 19 additions & 15 deletions server/src/routes/storage.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -154,48 +154,52 @@ function formatRunResponse(run: any) {
return formattedRun;
}

interface CredentialUpdate {
[selector: string]: string;
interface CredentialInfo {
value: string;
type: string;
}

function updateTypeActionsInWorkflow(workflow: any[], credentials: CredentialUpdate) {
interface Credentials {
[key: string]: CredentialInfo;
}

function updateTypeActionsInWorkflow(workflow: any[], credentials: Credentials) {
return workflow.map(step => {
if (!step.what) return step;

// First pass: mark indices to remove
const indicesToRemove = new Set<number>();
step.what.forEach((action: any, index: any) => {
step.what.forEach((action: any, index: number) => {
if (!action.action || !action.args?.[0]) return;

// If it's a type/press action for a credential
if ((action.action === 'type' || action.action === 'press') && credentials[action.args[0]]) {
indicesToRemove.add(index);
// Check if next action is waitForLoadState

if (step.what[index + 1]?.action === 'waitForLoadState') {
indicesToRemove.add(index + 1);
}
}
});

// Filter out marked indices and create new what array
const filteredWhat = step.what.filter((_: any, index: any) => !indicesToRemove.has(index));
const filteredWhat = step.what.filter((_: any, index: number) => !indicesToRemove.has(index));

// Add new type actions after click actions
Object.entries(credentials).forEach(([selector, credential]) => {
Object.entries(credentials).forEach(([selector, credentialInfo]) => {
const clickIndex = filteredWhat.findIndex((action: any) =>
action.action === 'click' && action.args?.[0] === selector
);

if (clickIndex !== -1) {
const chars = credential.split('');
const chars = credentialInfo.value.split('');

Copy link

@coderabbitai coderabbitai bot Jan 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Use Unicode-Safe String Splitting

When splitting strings that may contain Unicode characters, split('') may not handle surrogate pairs correctly. Use the spread operator [...string] or Array.from(string) to ensure all characters are handled properly.

Apply this diff to update the code:

-const chars = credentialInfo.value.split('');
+const chars = [...credentialInfo.value];

Committable suggestion skipped: line range outside the PR's diff.

chars.forEach((char, i) => {
// Add type action
filteredWhat.splice(clickIndex + 1 + (i * 2), 0, {
action: 'type',
args: [selector, encrypt(char)]
args: [
selector,
encrypt(char),
Copy link

@coderabbitai coderabbitai bot Jan 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Ensure Asynchronous Handling of encrypt Function

The encrypt function might be asynchronous, similar to how decrypt is used with await in decryptWorkflowActions. If encrypt returns a Promise, you need to handle it with await and modify the function accordingly.

Apply this diff to handle encrypt asynchronously:

-function updateTypeActionsInWorkflow(workflow: any[], credentials: Credentials) {
+async function updateTypeActionsInWorkflow(workflow: any[], credentials: Credentials) {
     return await Promise.all(workflow.map(async step => {
         if (!step.what) return step;

         const indicesToRemove = new Set<number>();
         step.what.forEach((action: any, index: number) => {
             // existing logic...
         });

         const filteredWhat = step.what.filter((_: any, index: number) => !indicesToRemove.has(index));

         await Promise.all(Object.entries(credentials).map(async ([selector, credentialInfo]) => {
             const clickIndex = filteredWhat.findIndex((action: any) =>
                 action.action === 'click' && action.args?.[0] === selector
             );

             if (clickIndex !== -1) {
                 const chars = [...credentialInfo.value];

                 await Promise.all(chars.map(async (char, i) => {
                     filteredWhat.splice(clickIndex + 1 + (i * 2), 0, {
                         action: 'type',
                         args: [
                             selector,
-                            encrypt(char),
+                            await encrypt(char),
                             credentialInfo.type
                         ]
                     });

                     filteredWhat.splice(clickIndex + 2 + (i * 2), 0, {
                         action: 'waitForLoadState',
                         args: ['networkidle']
                     });
                 }));
             }
         }));

         return {
             ...step,
             what: filteredWhat
         };
     }));
 }

Ensure that all calls to updateTypeActionsInWorkflow handle the Promise returned due to the function being asynchronous.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
encrypt(char),
async function updateTypeActionsInWorkflow(workflow: any[], credentials: Credentials) {
return await Promise.all(workflow.map(async step => {
if (!step.what) return step;
const indicesToRemove = new Set<number>();
step.what.forEach((action: any, index: number) => {
// existing logic...
});
const filteredWhat = step.what.filter((_: any, index: number) => !indicesToRemove.has(index));
await Promise.all(Object.entries(credentials).map(async ([selector, credentialInfo]) => {
const clickIndex = filteredWhat.findIndex((action: any) =>
action.action === 'click' && action.args?.[0] === selector
);
if (clickIndex !== -1) {
const chars = [...credentialInfo.value];
await Promise.all(chars.map(async (char, i) => {
filteredWhat.splice(clickIndex + 1 + (i * 2), 0, {
action: 'type',
args: [
selector,
await encrypt(char),
credentialInfo.type
]
});
filteredWhat.splice(clickIndex + 2 + (i * 2), 0, {
action: 'waitForLoadState',
args: ['networkidle']
});
}));
}
}));
return {
...step,
what: filteredWhat
};
}));
}

credentialInfo.type
]
});

// Add waitForLoadState
filteredWhat.splice(clickIndex + 2 + (i * 2), 0, {
action: 'waitForLoadState',
args: ['networkidle']
Expand Down
14 changes: 10 additions & 4 deletions server/src/workflow-management/classes/Generator.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -474,14 +474,18 @@ export class WorkflowGenerator {
public onKeyboardInput = async (key: string, coordinates: Coordinates, page: Page) => {
let where: WhereWhatPair["where"] = { url: this.getBestUrl(page.url()) };
const selector = await this.generateSelector(page, coordinates, ActionType.Keydown);

const elementInfo = await getElementInformation(page, coordinates, '', false);
const inputType = elementInfo?.attributes?.type || "text";

if (selector) {
where.selectors = [selector];
}
const pair: WhereWhatPair = {
where,
what: [{
action: 'press',
args: [selector, encrypt(key)],
args: [selector, encrypt(key), inputType],
}],
}
if (selector) {
Expand Down Expand Up @@ -992,6 +996,7 @@ export class WorkflowGenerator {
let input = {
selector: '',
value: '',
type: '',
actionCounter: 0,
};

Expand All @@ -1006,7 +1011,7 @@ export class WorkflowGenerator {
// when more than one press action is present, add a type action
pair.what.splice(index - input.actionCounter, input.actionCounter, {
action: 'type',
args: [input.selector, encrypt(input.value)],
args: [input.selector, encrypt(input.value), input.type],
}, {
action: 'waitForLoadState',
args: ['networkidle'],
Expand Down Expand Up @@ -1034,13 +1039,14 @@ export class WorkflowGenerator {
action: 'waitForLoadState',
args: ['networkidle'],
})
input = { selector: '', value: '', actionCounter: 0 };
input = { selector: '', value: '', type: '', actionCounter: 0 };
}
} else {
pushTheOptimizedAction(pair, index);
input = {
selector: condition.args[0],
value: condition.args[1],
type: condition.args[2],
actionCounter: 1,
};
}
Expand All @@ -1049,7 +1055,7 @@ export class WorkflowGenerator {
if (input.value.length !== 0) {
pushTheOptimizedAction(pair, index);
// clear the input
input = { selector: '', value: '', actionCounter: 0 };
input = { selector: '', value: '', type: '', actionCounter: 0 };
}
}
});
Expand Down
7 changes: 6 additions & 1 deletion src/api/storage.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,13 @@ import { ScheduleSettings } from "../components/robot/ScheduleSettings";
import { CreateRunResponse, ScheduleRunResponse } from "../pages/MainPage";
import { apiUrl } from "../apiConfig";

interface CredentialInfo {
value: string;
type: string;
}

interface Credentials {
[key: string]: string;
[key: string]: CredentialInfo;
}

export const getStoredRecordings = async (): Promise<string[] | null> => {
Expand Down
Loading