Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix a few flaky tests in several products (Auth, Analytics, Storage, RTDB, Firestore) #1520

Merged
merged 20 commits into from
Jan 22, 2024

Conversation

jonsimantov
Copy link
Contributor

@jonsimantov jonsimantov commented Jan 16, 2024

Fix some more tests that fail due to flakes, usually because of network instability.

Auth: Add retry in TestLinkAnonymousUserWithEmailCredential.
Analytics: Add retry to TestGetAnalyticsInstanceID, and Android retry to TestResettingGivesNewInstanceId.
Storage: Add additional timestamp logging and switch to verifying against remote timestamp, as FTL device local time is unpredictable.
Database: Switch to verifying against remote timestamp, as FTL device local time is unpredictable.
Firestore: Android tests are now timing out on FTL after 90 minutes, so bump up the FTL timeout by 30 more minutes.

Description

Provide details of the change, and generalize the change in the PR title above.


Testing

Describe how you've tested these changes. Link any manually triggered Integration tests or CPP binary SDK Packaging Github Action workflows, if applicable.


Type of Change

Place an x the applicable box:

  • Bug fix. Add the issue # below if applicable.
  • New feature. A non-breaking change which adds functionality.
  • Other, such as a build process or documentation change.

Notes

  • Bug fixes and feature changes require an update to the Release Notes section of release_build_files/readme.md.
  • Read the contribution guidelines CONTRIBUTING.md.
  • Changes to the public API require an internal API review. If you'd like to help us make Firebase APIs better, please propose your change in a feature request so that we can discuss it together.

@jonsimantov jonsimantov added tests-requested: quick Trigger a quick set of integration tests. skip-release-notes Skip release notes check labels Jan 16, 2024
@github-actions github-actions bot added tests: in-progress This PR's integration tests are in progress. and removed tests-requested: quick Trigger a quick set of integration tests. labels Jan 16, 2024
Copy link

github-actions bot commented Jan 16, 2024

Integration test with FLAKINESS (succeeded after retry)

Requested by @jonsimantov on commit 1f2089b
Last updated: Mon Jan 22 16:48 PST 2024
View integration test log & download artifacts

Failures Configs
analytics [TEST] [FLAKINESS] [Android] [1/3 os: ubuntu] [1/2 android_device: emulator_ftl_target]
(1 failed tests)  CRASH/TIMEOUT
firestore [TEST] [FLAKINESS] [Android] [1/3 os: windows] [1/2 android_device: emulator_ftl_target]
(1 failed tests)  ServerTimestampTest.TestServerTimestampsWorkViaSet

Add flaky tests to go/fpl-cpp-flake-tracker

@jonsimantov jonsimantov added the tests-requested: quick Trigger a quick set of integration tests. label Jan 16, 2024
@github-actions github-actions bot removed the tests-requested: quick Trigger a quick set of integration tests. label Jan 16, 2024
@jonsimantov jonsimantov changed the title Fix flaky tests on Auth, Analytics, and GMA (with additional debug output) Fix flaky tests on Auth and Analytics Jan 16, 2024
@jonsimantov jonsimantov added the tests-requested: full Trigger a FULL set of integration tests (uses expanded test matrix). label Jan 17, 2024
@github-actions github-actions bot added tests: failed This PR's integration tests failed. and removed tests-requested: full Trigger a FULL set of integration tests (uses expanded test matrix). labels Jan 17, 2024
@firebase-workflow-trigger firebase-workflow-trigger bot removed the tests: in-progress This PR's integration tests are in progress. label Jan 17, 2024
@jonsimantov jonsimantov changed the title Fix flaky tests on Auth and Analytics Fix flaky tests on Auth, Analytics, etc. Jan 17, 2024
@jonsimantov jonsimantov added the tests-requested: quick Trigger a quick set of integration tests. label Jan 17, 2024
@github-actions github-actions bot added tests: in-progress This PR's integration tests are in progress. tests: failed This PR's integration tests failed. and removed tests-requested: quick Trigger a quick set of integration tests. tests: failed This PR's integration tests failed. labels Jan 17, 2024
@jonsimantov jonsimantov added the tests-requested: quick Trigger a quick set of integration tests. label Jan 17, 2024
@github-actions github-actions bot added tests: failed This PR's integration tests failed. and removed tests-requested: quick Trigger a quick set of integration tests. tests: failed This PR's integration tests failed. labels Jan 17, 2024
@jonsimantov jonsimantov added the tests-requested: quick Trigger a quick set of integration tests. label Jan 17, 2024
@github-actions github-actions bot added the tests: failed This PR's integration tests failed. label Jan 19, 2024
@firebase-workflow-trigger firebase-workflow-trigger bot removed the tests: in-progress This PR's integration tests are in progress. label Jan 19, 2024
@jonsimantov jonsimantov added the tests-requested: quick Trigger a quick set of integration tests. label Jan 19, 2024
@github-actions github-actions bot added tests: in-progress This PR's integration tests are in progress. tests: succeeded This PR's integration tests succeeded. and removed tests-requested: quick Trigger a quick set of integration tests. tests: failed This PR's integration tests failed. labels Jan 19, 2024
@firebase-workflow-trigger firebase-workflow-trigger bot removed the tests: in-progress This PR's integration tests are in progress. label Jan 20, 2024
@jonsimantov jonsimantov added the tests-requested: quick Trigger a quick set of integration tests. label Jan 22, 2024
@github-actions github-actions bot added tests: in-progress This PR's integration tests are in progress. tests: failed This PR's integration tests failed. and removed tests-requested: quick Trigger a quick set of integration tests. tests: succeeded This PR's integration tests succeeded. labels Jan 22, 2024
@firebase-workflow-trigger firebase-workflow-trigger bot removed the tests: in-progress This PR's integration tests are in progress. label Jan 22, 2024
@jonsimantov jonsimantov added the tests-requested: quick Trigger a quick set of integration tests. label Jan 22, 2024
@github-actions github-actions bot added tests: in-progress This PR's integration tests are in progress. and removed tests-requested: quick Trigger a quick set of integration tests. tests: failed This PR's integration tests failed. labels Jan 22, 2024
@jonsimantov jonsimantov enabled auto-merge (squash) January 22, 2024 20:49
@github-actions github-actions bot added the tests: succeeded This PR's integration tests succeeded. label Jan 22, 2024
@firebase-workflow-trigger firebase-workflow-trigger bot removed the tests: in-progress This PR's integration tests are in progress. label Jan 22, 2024
@jonsimantov jonsimantov merged commit 1f2089b into main Jan 22, 2024
59 of 65 checks passed
@github-actions github-actions bot added tests: in-progress This PR's integration tests are in progress. tests: succeeded This PR's integration tests succeeded. and removed tests: succeeded This PR's integration tests succeeded. labels Jan 22, 2024
@firebase-workflow-trigger firebase-workflow-trigger bot removed the tests: in-progress This PR's integration tests are in progress. label Jan 23, 2024
@firebase firebase locked and limited conversation to collaborators Feb 22, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
skip-release-notes Skip release notes check tests: succeeded This PR's integration tests succeeded.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants