[Crashlyics] Address some Crashlytics flakes and disable others in nightly tests

[paulb777]

This PR should stabilize the Crashlytics unit tests after asking gemini to find and address several flaky tests. This PR doesn't touch the library itself other than exposing an additional property to the tests.

It also disables the flaky FIRCLSSettingsTests.m in nightly test runs.

This is a fresh start on #15551 which went too far with adding unnecessary churn and disabling some functionality.

This is an example of the common flake that will be suppressed in the nightly tests:

#no-changelog

Gemini's summary:

Summary of Changes

This pull request focuses on enhancing the stability and reliability of the Crashlytics unit test suite. It implements targeted fixes for existing flaky tests, such as improving file system interactions and cleanup procedures. Additionally, it introduces a configurable exclusion for certain flaky tests during nightly builds, aiming to prevent intermittent failures in automated pipelines. The changes also include minor refactoring to improve test code maintainability and thread safety in mock objects.

Highlights

• Test Stability Improvements: Several Crashlytics unit tests have been updated to address flakiness, ensuring more reliable test results by improving file system interactions and cleanup procedures.
• Flaky Test Exclusion: A mechanism has been introduced to conditionally exclude known flaky tests (specifically FIRCrashlyticsReportTests.m) from nightly test runs, preventing build failures due to intermittent issues.
• Mock File Manager Thread Safety: The mock file manager used in tests now includes thread-safe operations, which can help prevent race conditions in concurrent test environments.

Changelog

• Crashlytics/UnitTests/FIRCLSContextManagerTests.m
  • Ensures the root directory for the mock file manager is explicitly created during test setup.
• Crashlytics/UnitTests/FIRCLSExistingReportManagerTests.m
  • Modified setUp to ensure a clean state by recreating the file manager's root directory after removal.
• Crashlytics/UnitTests/FIRCLSReportUploaderTests.m
  • Implemented a polling loop with a timeout to wait for file removal, making the test more resilient to timing variations.
• Crashlytics/UnitTests/FIRCLSUserDefaultsTests.m
  • Added explicit removal of test-specific keys from NSUserDefaults in tearDown to prevent test interference.
• Crashlytics/UnitTests/Mocks/FIRCLSMockFileManager.m
  • Added @synchronized blocks to all methods that access or modify the internal fileSystemDict to ensure thread safety.
• Package.swift
  • Refactored the Crashlytics unit test target definition into a new function crashlyticsUnitTestChooser() to allow conditional exclusion of FIRCrashlyticsReportTests.m based on the CRASHLYTICS_NIGHTLY environment variable.

[gemini-code-assist]

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

[paulb777]

/gemini review

[gemini-code-assist]

Code Review

This pull request aims to stabilize Crashlytics unit tests by fixing some flaky tests and disabling others in nightly runs. The changes include better test setup/teardown hygiene, making the mock file manager thread-safe, and introducing a mechanism to disable a flaky test during nightly builds. My review focuses on improving the implementation of one of the flake fixes and reducing code duplication in the package manifest.

[paulb777]

/gemini summarize

[ncooke3]

Closing in favor of #15601. Thank you!