Skip to content

Conversation

@JamesNK
Copy link
Member

@JamesNK JamesNK commented May 21, 2025

Addresses #9214

@radical What's the process on fixing flaky tests? Do we leave them as flaky and check later to see if there are failures? If so, how do I do that?

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR enhances logging and improves test robustness for the console logs pause/resume functionality.

  • Updated the UI test to use WaitForElement and WaitForAssertion to help address test flakiness.
  • Added detailed debug logging in the console logs component during log load and pause/resume state changes.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
tests/Aspire.Dashboard.Components.Tests/Pages/ConsoleLogsTests.cs Updated test to properly wait for UI elements and assert pause state changes.
src/Aspire.Dashboard/Components/Pages/ConsoleLogs.razor.cs Added debug logs to provide better traceability when loading logs and updating pause state.

@radical
Copy link
Member

radical commented May 21, 2025

@radical What's the process on fixing flaky tests? Do we leave them as flaky and check later to see if there are failures? If so, how do I do that?

We mark any flaky tests with [QuarantinedTest(..)] attribute, and use a unique issue for it. The quarantined test runs are tracked in #8813 . The individual issues will have last 10 failures from the Quarantined test runs listed like in #8720 .

As for when to get them out of quarantine, I have been doing when there have been >100 passing runs. But that some of the dashboard tests failed after enabling them.

Note: there is a difference in how the quarantined tests are run. For a given test assembly, only the quarantined tests in that are run, instead of all the tests in the assembly.

Quarantined tests workflow - https://github.com/dotnet/aspire/actions/workflows/tests-outerloop.yml

@JamesNK
Copy link
Member Author

JamesNK commented May 21, 2025

Should I leave the attribute on the flaky test method and it will be removed later?

@JamesNK
Copy link
Member Author

JamesNK commented May 22, 2025

ping @adamint review please

@adamint adamint merged commit 4a88cb2 into main May 22, 2025
500 of 503 checks passed
@adamint adamint deleted the jamesnk/pause-flaky-test-again branch May 22, 2025 16:48
@github-actions github-actions bot locked and limited conversation to collaborators Jun 22, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants