Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test Reliability: Test infra should handle cases where test app is left in minimized state #1225

Closed
kmahone opened this issue Aug 27, 2019 · 0 comments · Fixed by #2311
Closed
Labels
area-TestInfrastructure Issue in the test infrastructure (e.g. in Helix scripts) team-Controls Issue for the Controls team

Comments

@kmahone
Copy link
Member

kmahone commented Aug 27, 2019

This CI build failed due to test failures:
https://dev.azure.com/ms/microsoft-ui-xaml/_build/results?buildId=33255&view=logs

The first failing test is:
AcrylicBrushTests.HideAndShowWindow
after that, the rest of the tests in that test class also fail. And they all also fail on re-run.

This issue is not about the specific test failure, but rather the test infra reliability issue that it exposes.

For some reason, this test crashed in the middle of execution. As a result of this, the machine was left in a configuration that it was unable to recover from:
The test app was left running, but minimized.

Issues:

  1. When we attempt to Close the app as part of cleanup via Application.Close, we fail to close the app due to not being able to find the appropriate UIA element ("Failed to find close app invoker:"), since the application is minimized. So after cleanup, the app is left running.
  2. When the next test in the test class attempts to run, it attempts to launch the application (Application.Initialize). The app is already running, so it does not attempt to launch it. But it cannot find its CoreWindow (since the app is minimized), so it fails.
  3. When the te.exe run is complete, we attempt to re-run the failed tests. Even though we are starting with a new clean process, etc. the test app is still running and minimized, so we hit the same issue in NavigationView TogglePaneTopPadding issue #2 again for all the re-runs.

Fixes:

  1. As part of Application.Close we should try to terminate the process if we cannot close it via UIA.
  2. As part of Application.Initialize, we should try to maximize the top level window, before trying to find the CoreWindow that is a child of it,
  3. In the runtests.cmd script, we should ensure that we terminate any left over test processes before we do any re-runs.
@jevansaks jevansaks added the area-TestInfrastructure Issue in the test infrastructure (e.g. in Helix scripts) label Aug 27, 2019
@jevansaks jevansaks added the team-Controls Issue for the Controls team label Nov 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-TestInfrastructure Issue in the test infrastructure (e.g. in Helix scripts) team-Controls Issue for the Controls team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants