Investigate workflow job failing on main: e2ePerformanceTests / Run E2E tests in AWS device farm

[github-actions]

🚨 Failure Summary 🚨:

• 📋 Job Name: e2ePerformanceTests / Run E2E tests in AWS device farm
• 🔧 Failure in Workflow: Process new code merged to main
• 🔗 Triggered by PR: PR Link
• 👤 PR Author: @christianwen
• 🤝 Merged by: @iwiznia
• 🐛 Error Message:
  warning: No matching artifact of type LOG found for debug.log, skipping
  failure: Test run failed after 3090 seconds with: undefined. Timeout is set to 7200
  warning: No matching artifact of type LOG found for debug.log, skipping
  failure: Test run failed after 2915 seconds with: undefined. Timeout is set to 7200
  failure: Process returned exit code 1
  Launched as "/home/runner/runners/2.321.0/externals/node20/bin/node /home/runner/work/_actions/Wandalen/wretry.action/v3.5.0_js_action/src/Runner.mjs /home/runner/work/_actions/Wandalen/aws-devicefarm/test-application/dist/index.js"
  Launched at "/home/runner/work/App/App"
  Attempts exhausted, made 2 attempts :
  Attempt Some initial fixes and code style updates #1 started at : 18:06:37 GMT+0000 (Coordinated Universal Time)
  Attempt Fix spaces #2 started at : 18:59:47 GMT+0000 (Coordinated Universal Time)
  failure: Process completed with exit code 1.

⚠️ Action Required ⚠️:

🛠️ A recent merge appears to have caused a failure in the job named e2ePerformanceTests / Run E2E tests in AWS device farm.
This issue has been automatically created and labeled with Workflow Failure for investigation.

👀 Please look into the following:

1. Why the PR caused the job to fail?
2. Address any underlying issues.

🐛 We appreciate your help in squashing this bug!

[iwiznia]

@christianwen @DylanDylann got this issue assigned, should I have not merged the PR #51172?

[christianwen]

@iwiznia I think the error log here is unrelated to my PR and I can run this job on my local device.

Warning: No matching artifact of type LOG found for debug.log, skipping
Error: Test run failed after 2915 seconds with: undefined. Timeout is set to 7200

I see there're 2 PRs that fails job:

• Investigate workflow job failing on main: test / test (job 3) #54547
• Investigate workflow job failing on main: confirmPassingBuild #54548

And re-run successfully. Should we give it another try?

[melvin-bot]

📣 @christianwen! 📣
Hey, it seems we don’t have your contributor details yet! You'll only have to do this once, and this is how we'll hire you on Upwork.
Please follow these steps:

1. Make sure you've read and understood the contributing guidelines.
2. Get the email address used to login to your Expensify account. If you don't already have an Expensify account, create one here. If you have multiple accounts (e.g. one for testing), please use your main account email.
3. Get the link to your Upwork profile. It's necessary because we only pay via Upwork. You can access it by logging in, and then clicking on your name. It'll look like this. If you don't already have an account, sign up for one here.
4. Copy the format below and paste it in a comment on this issue. Replace the placeholder text with your actual details.
   
   Format:

Contributor details
Your Expensify account email: <REPLACE EMAIL HERE>
Upwork Profile Link: <REPLACE LINK HERE>

[iwiznia]

And re-run successfully. Should we give it another try?

It failed again https://github.com/Expensify/App/actions/runs/12484590267/job/34890814512

[iwiznia]

@christianwen @DylanDylann can you take a look please?

[iwiznia]

Bump @christianwen @DylanDylann

[DylanDylann]

I just checked it but still struggle with this one. @christianwen Do you have any input on it?

[christianwen]

I have no idea about this job, can we run it locally @iwiznia ?

[iwiznia]

I am not sure honestly, it seems it runs in aws device farm, but maybe you can run it locally.

[DylanDylann]

Hmmm, It is so strange to me. I don't see any valuable log that can help investigate this. @iwiznia Could you ask someone who is familiar with this flow to support us?

[iwiznia]

Oh I thought you were familiar with it, since you said this. Can you post it in slack to get help from people more familiar with this?

[DylanDylann]

@iwiznia If that is the reason, I think we can ignore it because it is intentional. Is there any problem with leaving it?

[iwiznia]

Ignore what? The fact that we made this fail? I don't think we can or it will keep failing, no?

[DylanDylann]

I am trying to simulate this test locally. I will try to give an update tomorrow

[christianwen]

I'm also trying to mimic the test. Will give an update by EOD

[iwiznia]

ok cool, let me know

[DylanDylann]

@iwiznia Do we have any document/guideline for e2ePerformanceTests jobs? It seems we need to simulate an environment on AWS to run it locally

@christianwen Do you have any update here?

[DylanDylann]

@kirillzyusko I noticed you already worked on this area recently #51182. Could you help on this one?

[christianwen]

@DylanDylann @kirillzyusko I'm stuck on the way to simulate this test on local machine. It would be great if we have the instruction for this.

[iwiznia]

@iwiznia Do we have any document/guideline for e2ePerformanceTests jobs? It seems we need to simulate an environment on AWS to run it locally

Not really, but I assume someone must have it, post it in the public slack channel (and then post the link to that here) and I bet someone that knows this will help you out.

[DylanDylann]

The Slack link: https://expensify.slack.com/archives/C01GTK53T8Q/p1736483119612509

[kirillzyusko]

I think the problem is solved in #55000 right?

[DylanDylann]

@iwiznia Could you please run this job again?

[iwiznia]

Re-run here https://github.com/Expensify/App/actions/runs/12484590267 but not sure if that's what we want since I think that will run it with the old code... 🤔