Skip to content

Chore: improving base case create project and settle in playwright test.#6248

Closed
nadr0 wants to merge 3 commits intomainfrom
nadro/create-project-stress-test
Closed

Chore: improving base case create project and settle in playwright test.#6248
nadr0 wants to merge 3 commits intomainfrom
nadro/create-project-stress-test

Conversation

@nadr0
Copy link
Contributor

@nadr0 nadr0 commented Apr 10, 2025

Issue

running yarn test:playwright:electron:local -g "Create project and load stress test" --repeat-each="10" would fail 30% of the time.

Fix

Increase the timeout on the settle for the disabled button.

It always failed on

    Locator: getByRole('button', { name: 'Start Sketch' }).or(getByRole('button', { name: 'Edit Sketch' }))
    Expected: not disabled
    Received: disabled
    Call log:
      - expect.not.toBeDisabled with timeout 5000ms
      - waiting for getByRole('button', { name: 'Start Sketch' }).or(getByRole('button', { name: 'Edit Sketch' }))
        5 × locator resolved to <button disabled id="sketch" tabindex="-1" name="Start Sketch" data-testid="sketch" aria-description="Start drawing a 2D sketch" class="action-button p-0 m-0 group mono text-xs leading-none flex items-center gap-2 rounded-sm border-solid border border-chalkboard-30 hover:border-chalkboard-40 enabled:dark:border-chalkboard-70 dark:hover:border-chalkboard-60 dark:bg-chalkboard-90/50 text-chalkboard-100 dark:text-chalkboard-10 pr-2 pressed:!text-chalkboard-10 pressed:enabled:hovered:!text-chalkboard-10 !…>…</button>
          - unexpected value "disabled"

I did a massive debug print statement to see isExeccuting, disableAllButtons, and the engine connection state

Tests

With 15_000 timeout.

yarn test:playwright:electron:local -g "Create project and load stress test" --repeat-each="30"
30 passed (10.3m)

@qa-wolf
Copy link

qa-wolf bot commented Apr 10, 2025

QA Wolf here! As you write new code it's important that your test coverage is keeping up.
Click here to request test coverage for this PR!

@vercel
Copy link

vercel bot commented Apr 10, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated (UTC)
modeling-app ✅ Ready (Inspect) Visit Preview Apr 10, 2025 4:24pm

Copy link
Contributor

@Irev-Dev Irev-Dev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

15 seconds is pretty long.

Are there more underlying issues do you think, or it really just needs this long?

@codecov
Copy link

codecov bot commented Apr 10, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 84.90%. Comparing base (970cf7f) to head (a283f5a).
Report is 3 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #6248   +/-   ##
=======================================
  Coverage   84.90%   84.90%           
=======================================
  Files         109      109           
  Lines       44842    44842           
=======================================
  Hits        38073    38073           
  Misses       6769     6769           
Flag Coverage Δ
rust 84.90% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@franknoirot
Copy link
Contributor

Hey this got rolled into #6265, and it got things a little more solidly green. Nice work!

@nadr0 nadr0 deleted the nadro/create-project-stress-test branch June 2, 2025 21:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants