Skip to content

Improve Test Filter Management#2944

Closed
sol-loup wants to merge 1 commit intomainfrom
safe_global_filter_test_class
Closed

Improve Test Filter Management#2944
sol-loup wants to merge 1 commit intomainfrom
safe_global_filter_test_class

Conversation

@sol-loup
Copy link
Copy Markdown
Contributor

@sol-loup sol-loup commented Mar 13, 2025

PR: Improve Test Filter Management with AbstractWPUnitTestWithSafeFiltering

This PR addresses a common source of test flakiness by implementing proper WordPress filter management in our test suite.

The Problem

Sometimes when testing, you want to override the behavior of a global function. This can be done using a wordpress primitive called add_filter

While this can be helpful for mocking deep functionality, these filters present a challenge when testing across hundreds of files:

  1. Filter Leakage: When add_filter() is used in tests without proper cleanup, filters persist between tests, causing unexpected behavior and interdependencies.

  2. Incomplete Cleanup: Manual filter removal is error-prone and often forgotten, especially when tests fail before reaching teardown code.

  3. Overly Aggressive Cleanup: Using remove_all_filters() is dangerous as it removes ALL filters for a hook, including those added by WordPress core or other plugins, potentially breaking functionality in subsequent tests.

Solution Proposed by this PR

The AbstractWPUnitTestWithSafeFiltering class provides:

  • A add_filter_with_safe_teardown() method that tracks all added filters
  • Automatic cleanup in tearDown() that only removes filters added by the current test
  • The ability to remove specific filters early with teardown_safely_immediately()
  • A category-specific cleanup method with teardown_callback_category_safely() -- allowing you to safely wipe out all test-related filters for 'event_type_A' without affecting B or C.

Changes

  • Introduced AbstractWPUnitTestWithSafeFiltering with above functionality.
  • Updated WCFacebookCommerceIntegrationTest, fbproductTest, WPMLTest, and FeedUploadUtilsTest to extend AbstractWPUnitTestWithSafeFiltering
  • Replaced all add_filter() calls with add_filter_with_safe_teardown()
  • in some special cases (per-test), replaced remove_all_filters with appropriate teardown calls
  • In most cases, rely on default "invisible" safe teardown behavior

Testing Performed

  • Ran updated test suit using phpunit -- all succeeded.

@facebook-github-bot
Copy link
Copy Markdown
Contributor

@sol-loup has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Copy Markdown
Contributor

@sol-loup merged this pull request in d61751e.

@sol-loup sol-loup mentioned this pull request Mar 25, 2025
7 tasks
sharunaanandraj pushed a commit that referenced this pull request May 14, 2025
Summary:

This PR addresses a common source of test flakiness by implementing proper WordPress filter management in our test suite.

Sometimes when testing, you want to override the behavior of a global function. This can be done using a wordpress primitive called `add_filter`

While this can be helpful for mocking deep functionality, these filters present a challenge when testing across hundreds of files:

1. **Filter Leakage**: When `add_filter()` is used in tests without proper cleanup, filters persist between tests, causing unexpected behavior and interdependencies.

2. **Incomplete Cleanup**: Manual filter removal is error-prone and often forgotten, especially when tests fail before reaching teardown code.

3. **Overly Aggressive Cleanup**: Using `remove_all_filters()` is dangerous as it removes ALL filters for a hook, including those added by WordPress core or other plugins, potentially breaking functionality in subsequent tests.

The `AbstractWPUnitTestWithSafeFiltering` class provides:

- A `add_filter_with_safe_teardown()` method that tracks all added filters
- Automatic cleanup in `tearDown()` that only removes filters added by the current test
- The ability to remove specific filters early with `teardown_safely_immediately()`
- A category-specific cleanup method with `teardown_callback_category_safely()` -- allowing you to safely wipe out all test-related filters for 'event_type_A' without affecting B or C.

- Introduced AbstractWPUnitTestWithSafeFiltering with above functionality.
- Updated WCFacebookCommerceIntegrationTest, fbproductTest, WPMLTest, and FeedUploadUtilsTest to extend AbstractWPUnitTestWithSafeFiltering
- Replaced all `add_filter()` calls with `add_filter_with_safe_teardown()`
- in some special cases (per-test), replaced remove_all_filters with appropriate teardown calls
- In most cases, rely on default "invisible" safe teardown behavior

- Ran updated test suit using phpunit -- all succeeded.

Pull Request resolved: #2944

Test Plan:
Imported from GitHub, without a `Test Plan:` line.

**!---- (auto-generated) DO NOT EDIT OR PUT ANYTHING AFTER THIS LINE ----!**
MFTRunTestsScript Run / Test Suite: sa_checkout / Test Collection: bloks / Diff Version V1
https://internalfb.com/intern/testinfra/testrun/1970325109964811
MFTRunTestsScript Run / Test Suite: sa_checkout / Test Collection: www / Diff Version V1
https://internalfb.com/intern/testinfra/testrun/3659174967893694

Reviewed By: carterbuce

Differential Revision: D71158932

Pulled By: sol-loup

fbshipit-source-id: 8b3479374bb836dd74a742905226b285da2bfe66
@tzahgr tzahgr mentioned this pull request May 20, 2025
@carterbuce carterbuce mentioned this pull request May 29, 2025
@tzahgr tzahgr mentioned this pull request Jun 4, 2025
facebook-github-bot pushed a commit that referenced this pull request Jun 4, 2025
Summary:
= 3.5.0 - 2025-05-28 =
*  Add - Create basic checkout permalink w/ products and coupon support by ajello-meta in #2887
*  Add - Common Feed Upload Framework by Jmencab in #2875
*  Fix - Fix bug where templates were not loading correctly by ajello-meta in #2915
*  Tweak - Change MICE to use base site url instead of shop url by carterbuce in #2934
*  Tweak - Improve custom checkout UI by ajello-meta in #2930
*  Tweak - Make custom checkout UI mobile compatible by ajello-meta in #2942
*  Fix - Update parsing for Checkout URL Product IDs by carterbuce in #2935
*  Add - Implement dummy logging util by nealweiMeta in #2920
*  Add - Setup cron job for batch logging with global message queue by nealweiMeta in #2924
*  Add - Error log request api activate by nealweiMeta in #2933
*  Add - Log locally with debug mode enabled by nealweiMeta in #2939
*  Add - Ratings and reviews feed upload by nrostrow-meta in #2937
*  Tweak - Feed upload skip logic and logging calls by nrostrow-meta in #2964
*  Add - Add function to fetch feed upload instance by nrostrow-meta in #2970
*  Tweak - Have feed uploads always use feed generator by nrostrow-meta in #2971
*  Tweak - Trigger metadata feed uploads on CPI ID change (post onboarding) by nrostrow-meta in #2995
*  Add - Shipping profile feed upload button by nrostrow-meta in #3140
*  Add - Navigation menu feed upload logic by nrostrow-meta in #3159
*  Fix - Fixing some fclose and logging gaps in the feed upload logic by nrostrow-meta in #3192
*  Add - Enabling navigation menu feed upload and adding manual sync button by nrostrow-meta in #3223
*  Add - Promotions feed upload by carterbuce in #2941
*  Add - Plugin AJAX API Framework by sol-loup in #2928
*  Tweak - Test Infrastructure Enhancement by sol-loup in #2944
*  Add - Implement telemetry logs api by nealweiMeta in #2940
*  Fix -  Make error logging event configurable by nealweiMeta in #2954
*  Add - Implement logging toggle by nealweiMeta in #2959
*  Fix - auto products sync by nealweiMeta in #2978
*  Tweak - Sync products with restriction by nealweiMeta in #2983
*  Fix - Fix use_enhanced_onboarding for legacy connections by carterbuce in #2986
*  Add - Create enhanced settings UI by ajello-meta in #2968
*  Add - Create new troubleshooting drawer from legacy debug settings by ajello-meta in #2977
*  Add - Add manual product and coupon sync buttons by ajello-meta in #2984
*  Tweak - Make page title in enhanced settings static by ajello-meta in #2985
*  Tweak - Align finalized content for logging toggle by nealweiMeta in #2992
*  Tweak - Improve local log by nealweiMeta in #3009
*  Fix - Fix free shipping coupon sync by carterbuce in #2993
*  Tweak - Add logging for feed generation scheduling failure by carterbuce in #2994
*  Tweak - Add logging in checkout for coupon code by ajello-meta in #2991
*  Tweak - Clean up CSS in enhanced settings UI by ajello-meta in #2996
*  Tweak - Remove the "Advertise" tab by ajello-meta in #3024
*  Tweak - Sync "Usage Count" in Promos Feed by carterbuce in #3036
*  Tweak - Disable mini_shops product capability for unsupported items by carterbuce in #3084
*  Add - Add usage logging for enhanced settings tabs by ajello-meta in #3202
*  Tweak - Remove UI of a checkbox that controls enablement of the new style feed generation by mshymon in #3056
*  Fix - Fix linter errors for ./includes/fbutils.php files by ajello-meta in #3075
*  Fix - Hotfix for Rollout Switches by vinkmeta in #3236
*  Add - Opt out sync experience. by SayanPandey in #3220
*  Fix - Added a transient flag to avoid flooding of product set api requests by vinkmeta in #3245
*  Fix - Additional check for the opt-out banner by SayanPandey in #3259
*  Fix - Bump up GraphAPI version to 21 by vahidkay-meta in #3219
*  Fix - fix linter errors for ./class-wc-facebookcommerce.php by ajello-meta in #3255
*  Fix - fix linter errors for ./facebook-commerce-events-tracker.php by ajello-meta in #3254
*  Fix - fix linter errors for ./includes/Admin/Settings_Screens/Advertise.php by ajello-meta in #3237
*  Fix - fix linter errors for ./includes/Admin/Settings_Screens/Product_Sync.php by ajello-meta in #3239
*  Fix - fix function return typing for get_settings() by ajello-meta in #3257
*  Tweak - Addition check for opt out by SayanPandey in #3259
*  Tweak - Update the GraphAPI version to 21 by vahidkay-meta in #3219
*  Fix - Enabled rollout switch only for plugin admins by vinkmeta in #3242
*  Add - reset connection functionality by jczhuoMeta in #3262
*  Fix - fixing the non static method called as static issue by SayanPandey in #3263
*  Fix - Fix linter errors for ./facebook-commerce.php by ajello-meta in #3251

Pull Request resolved: #3264

Test Plan:
Imported from GitHub, without a `Test Plan:` line.

**!---- (auto-generated) DO NOT EDIT OR PUT ANYTHING AFTER THIS LINE ----!**
MFTRunTestsScript Run / Test Suite: sa_checkout / Test Collection: bloks / Diff Version V7
https://internalfb.com/intern/testinfra/testrun/10414574226197703
MFTRunTestsScript Run / Test Suite: sa_checkout / Test Collection: www / Diff Version V7
https://internalfb.com/intern/testinfra/testrun/844425351364266

Reviewed By: vinkmeta, nrostrow-meta

Differential Revision: D75569981

Pulled By: carterbuce

fbshipit-source-id: 11bb39a9d66dfcbc80af413466feef0792d27ce4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants