fix(tests): fix or ignore unstable tests by shamardy · Pull Request #2365 · GLEECBTC/komodo-defi-framework

shamardy · 2025-02-18T13:07:32Z

No description provided.

…sender_eth` test

…isable `test_qrc20_tx_history` in wasm

…` in windows and update `TBTC_ELECTRUMS`

borngraced

I think simply ignoring the test might be a better option compared to commenting them out, compiler won't catch any changes that might affect the commented tests in the feature (if in any case)

shamardy · 2025-02-19T12:51:10Z

I think simply ignoring the test might be a better option compared to commenting them out, compiler won't catch any changes that might affect the commented tests in the feature (if in any case)

How do you ignore wasm_bindgen_test? this PR is still in progress either way as I will open issues for the ignored / commented out tests before making it ready for review.

borngraced · 2025-02-19T13:01:23Z

I think simply ignoring the test might be a better option compared to commenting them out, compiler won't catch any changes that might affect the commented tests in the feature (if in any case)

How do you ignore wasm_bindgen_test? this PR is still in progress either way as I will open issues for the ignored / commented out tests before making it ready for review.

apparently, we can't ignore wasm_bindgen tests..commenting is the only way I think.

shamardy · 2025-02-19T13:06:05Z

apparently, we can't ignore wasm_bindgen tests..commenting is the only way I think.

They are the only tests that are commented out for this reason :)

shamardy · 2025-02-19T13:23:53Z

Maybe we can let the wasm_bindgen_test print something in the logs and we comment out the test body, what do you think @borngraced?

Testing this out with the latest commit

Edit:
I think this is better than nothing
https://github.com/KomodoPlatform/komodo-defi-framework/actions/runs/13413866520/job/37469861883#step:9:473
https://github.com/KomodoPlatform/komodo-defi-framework/actions/runs/13413866520/job/37469861883#step:9:512
https://github.com/KomodoPlatform/komodo-defi-framework/actions/runs/13413866520/job/37469861883#step:9:642

borngraced · 2025-02-19T17:41:42Z

mm2src/mm2_net/src/native_http.rs

+        // The purpose of this test is to verify that the `slurp_url` function successfully retrieves data from an external service.
+        // This retry loop was added to mitigate any test instability,
+        // since the target service (https://httpbin.org/get) can sometimes return a temporary failure (e.g., 502 Bad Gateway).
+        for _ in 0..6 {
+            let (status, headers, body) =
+                block_on(slurp_url("https://httpbin.org/get")).expect("Request failed unexpectedly");
+            if status.is_success() {
+                return;
+            }
+            log!(
+                "Request failed with status code: {}, headers: {:?}, body: {}, retrying...",
+                status,
+                headers,
+                String::from_utf8_lossy(&body)
+            );
+            block_on(Timer::sleep(5.));
+        }
+
+        panic!("Failed to retrieve data from the external service after multiple attempts. Please check the service status.");


we can revert this change and depend on https://postman-echo.com/get uptime seems very okay

shamardy · 2025-02-19T23:03:59Z

In this PR, the CI has been very stable since this commit eff9652. Any failures were due to a cancellation triggered by pushing a new commit. In total, about 11 commits have had their testing jobs successfully completed. There may still be occasional flaky tests in the future, but most should be resolved by re-running the failed jobs. If that doesn’t help, we’ll look for additional SPV servers or disable the problematic tests on the relevant machine, at least until a more permanent solution can be implemented.

borngraced

Great work!

onur-ozkan · 2025-02-20T06:43:10Z

mm2src/mm2_p2p/src/behaviours/mod.rs


    #[tokio::test]
-    #[cfg(not(windows))] // https://github.com/KomodoPlatform/atomicDEX-API/issues/1712
+    #[cfg(target_os = "linux")] // https://github.com/KomodoPlatform/atomicDEX-API/issues/1712


Why only linux OS instead of UNIX platform?

I guess it fails on MacOS too. Can we do "unix but not macos"?

Thanks, will look to fix it in another PR. Will add this to the nextest issue

mariocynicys · 2025-02-20T08:40:24Z

They are the only tests that are commented out for this reason :)

could we have a flakey feature flag and do cfg(flakey) around them instead of commenting? we can use that feature flag for other platforms also.

shamardy · 2025-02-20T09:16:50Z

could we have a flakey feature flag and do cfg(flakey) around them instead of commenting? we can use that feature flag for other platforms also.

Will open an issue for nextest and add this idea ref. https://nexte.st/docs/features/retries/#per-test-settings

* dev: (24 commits) fix(eth-tpu): remove state from funding validation (GLEECBTC#2334) improvement(rpc-server): rpc server dynamic port allocation (GLEECBTC#2342) fix(tests): fix or ignore unstable tests (GLEECBTC#2365) fix(fs): make `filter_files_by_extension` return only files (GLEECBTC#2364) fix(derive_key_from_path): check length of current_key_material (GLEECBTC#2356) chore(release): bump mm2 version to 2.4.0-beta (GLEECBTC#2346) fix(tests): add additional testnet sepolia nodes to test code (GLEECBTC#2358) fix(swaps): maintain legacy compatibility for negotiation messages (GLEECBTC#2353) refactor(SwapOps): impl defaults for protocol specific swapops fns (GLEECBTC#2354) feat(tpu-v2): provide swap protocol versioning (GLEECBTC#2324) feat(wallet): add change mnemonic password rpc (GLEECBTC#2317) fix(tpu-v2): fix tpu-v2 wait for payment spend and extract secret (GLEECBTC#2261) feat(tendermint): unstaking/undelegation (GLEECBTC#2330) fix(utxo-withdraw): get hw ctx only when `PrivKeyPolicy` is trezor (GLEECBTC#2333) feat(event-streaming): API-driven subscription management (GLEECBTC#2172) fix(hash-types): remove panic, enforce fixed-size arrays (GLEECBTC#2279) fix(ARRR): store unconfirmed change output (GLEECBTC#2276) feat(tendermint): staking/delegation (GLEECBTC#2322) chore(deps): `timed-map` migration (GLEECBTC#2247) fix(mem-leak): `running_swap` never shrinks (GLEECBTC#2301) ...

* dev: feat(tendermint): claim delegation rewards (#2351) fix(eth-tpu): remove state from funding validation (#2334) improvement(rpc-server): rpc server dynamic port allocation (#2342) fix(tests): fix or ignore unstable tests (#2365) fix(fs): make `filter_files_by_extension` return only files (#2364)

* dev: feat(rpc): add is_success field to legacy MySwapStatusResponse (#2371) fix(key-derivation): use stored Argon2 parameters instead of default values (#2360) fix(tests): stabilize `tendermint_coin::test_claim_staking_rewards` (#2373) improvement(RPCs): group staking rpcs under a namespace (#2372) feat(tendermint): claim delegation rewards (#2351) fix(eth-tpu): remove state from funding validation (#2334) improvement(rpc-server): rpc server dynamic port allocation (#2342) fix(tests): fix or ignore unstable tests (#2365) fix(fs): make `filter_files_by_extension` return only files (#2364) fix(derive_key_from_path): check length of current_key_material (#2356) chore(release): bump mm2 version to 2.4.0-beta (#2346) fix(tests): add additional testnet sepolia nodes to test code (#2358) fix(swaps): maintain legacy compatibility for negotiation messages (#2353) refactor(SwapOps): impl defaults for protocol specific swapops fns (#2354) feat(tpu-v2): provide swap protocol versioning (#2324) feat(wallet): add change mnemonic password rpc (#2317)

add more eth mainnet nodes to fix unstable `validate_dex_fee_invalid_…

dfbf994

…sender_eth` test

shamardy added the status: in progress label Feb 18, 2025

shamardy added 12 commits February 18, 2025 15:46

ignore some unstable tests completely or for some OSs

8407a9f

fix windows fmt

277123d

increase wasm tests time, disable more tests for some OSs

b690385

fix fmt

55e252d

disable test_periodic_streamer on macos since it's unstable, also d…

01b6ee2

…isable `test_qrc20_tx_history` in wasm

disable test_until_success for macOS

f75eb91

try to fix unstable test_remove_delegation_qtum

eff9652

fix typo in tqtum electrum url

e53c499

remove qtum_electrums function as it used tqtum electrums urls

46a0da6

bring back test_qrc20_tx_history for wasm

fd8f309

bring back test_wait_for_confirmations_exceptedin windows

1d74f13

bring back test_tx_history_segwit, `test_tx_history_tbtc_non_segwit…

1810d75

…` in windows and update `TBTC_ELECTRUMS`

borngraced suggested changes Feb 19, 2025

View reviewed changes

shamardy mentioned this pull request Feb 19, 2025

Fix wasm zcoin tests failures due to light client unavailability #2366

Open

print warnings for skipped wasm tests

9a56910

shamardy mentioned this pull request Feb 19, 2025

Fix nft::nft_tests::test_camo unstability #2367

Closed

fix comment on test_camo

0a61d59

shamardy force-pushed the ignore-unstable-tests branch from bab795f to 0a61d59 Compare February 19, 2025 15:38

shamardy mentioned this pull request Feb 19, 2025

incompatible and unstable implementations/tests #1712

Open

shamardy added 2 commits February 19, 2025 17:58

remove comment from test_request_response_ok_three_peers

9cea6e3

make test_slurp_req more stable

79d59a7

shamardy force-pushed the ignore-unstable-tests branch from 85e435e to 79d59a7 Compare February 19, 2025 17:30

borngraced suggested changes Feb 19, 2025

View reviewed changes

move error comments to related issues

97d9e0a

revert test_slurp_req changes and use a different more stable url

da7ddad

shamardy added status: pending review and removed status: in progress labels Feb 19, 2025

borngraced approved these changes Feb 19, 2025

View reviewed changes

shamardy merged commit a559236 into dev Feb 19, 2025

shamardy deleted the ignore-unstable-tests branch February 19, 2025 23:19

onur-ozkan reviewed Feb 20, 2025

View reviewed changes

laruh mentioned this pull request Feb 23, 2025

fix(tests): make zombie tests working #2369

Closed

shamardy mentioned this pull request Apr 16, 2025

chore(release): v2.4.0-beta #2345

Merged

Conversation

shamardy commented Feb 18, 2025

Uh oh!

borngraced left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shamardy commented Feb 19, 2025

Uh oh!

borngraced commented Feb 19, 2025

Uh oh!

shamardy commented Feb 19, 2025

Uh oh!

shamardy commented Feb 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

borngraced Feb 19, 2025

Choose a reason for hiding this comment

Uh oh!

shamardy Feb 19, 2025

Choose a reason for hiding this comment

Uh oh!

shamardy commented Feb 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

borngraced left a comment

Choose a reason for hiding this comment

Uh oh!

onur-ozkan Feb 20, 2025

Choose a reason for hiding this comment

Uh oh!

onur-ozkan Feb 20, 2025

Choose a reason for hiding this comment

Uh oh!

shamardy Feb 20, 2025

Choose a reason for hiding this comment

Uh oh!

mariocynicys commented Feb 20, 2025

Uh oh!

shamardy commented Feb 20, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

borngraced left a comment •

edited

Loading

shamardy commented Feb 19, 2025 •

edited

Loading

shamardy commented Feb 19, 2025 •

edited

Loading