Skip to content

Stabilitze pov-recovery zombienet-sdk test#9007

Merged
lrubasze merged 4 commits intoci/zombienet-cumulus-tests-zombienet-sdkfrom
skunert/simplify-pov-recovery
Jun 27, 2025
Merged

Stabilitze pov-recovery zombienet-sdk test#9007
lrubasze merged 4 commits intoci/zombienet-cumulus-tests-zombienet-sdkfrom
skunert/simplify-pov-recovery

Conversation

@skunert
Copy link
Copy Markdown
Contributor

@skunert skunert commented Jun 27, 2025

closes #8985

I examined the two highlighted failures in the issue. In two cases, node three fails to recover the required blocks.

In the first case, the node is able to recover 16 blocks before encountering:

2025-06-24 06:00:12.022 ERROR ⋮cumulus-relay-chain-streams: [Parachain] Failed to fetch pending candidates. error=JsonRpcError(Call(ErrorObject { code: ServerError(-32000), message: "Error when retrieving the call proof: Call proof query errors:\n- ConnectionShutdown", data: None }))
2025-06-24 06:00:12.023 ERROR ⋮cumulus-relay-chain-streams: [Parachain] Failed to fetch session index. error=JsonRpcError(Call(ErrorObject { code: ServerError(-32000), message: "Error when retrieving the call proof: No node available for call proof query", data: None }))

For the other failure we also see the same RPC-related error. Since for node three, the RPC is the internal light client, I just disabled it and purged it from this test.

We will probably remove support for this experimental feature at some point, but currently does not have high priority. And should for sure not impact the stability of this test.

@skunert skunert requested review from lrubasze and pepoviola June 27, 2025 07:55
@skunert skunert added R0-no-crate-publish-required The change does not require any crates to be re-published. T10-tests This PR/Issue is related to tests. T18-zombienet_tests Trigger zombienet CI tests. labels Jun 27, 2025
@paritytech-workflow-stopper
Copy link
Copy Markdown

All GitHub workflows were cancelled due to failure one of the required jobs.
Failed workflow url: https://github.com/paritytech/polkadot-sdk/actions/runs/15921207310
Failed job name: run-frame-omni-bencher

@pepoviola
Copy link
Copy Markdown
Contributor

Looks great, thanks for the fix @skunert 🙌 . Make sense to add a new issue to track this

We will probably remove support for this experimental feature at some point, but currently does not have high priority. And should for sure not impact the stability of this test.

Thx!

@skunert
Copy link
Copy Markdown
Contributor Author

skunert commented Jun 27, 2025

Yep, issue now here #9013

Copy link
Copy Markdown
Contributor

@lrubasze lrubasze left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! The simpler the more stable 😁

@lrubasze
Copy link
Copy Markdown
Contributor

lrubasze commented Jun 27, 2025

BTW these failing Cumulus CI tests are not related to these changes.
Some regression introduced after merging master yesterday (most likely)

EDIT!
To clarify.
Yesterday I've merged master to ci/zombienet-cumulus-tests-zombienet-sdk.
One of the change was the runner change: parity-large (16 cores) -> parity-zombienet-native-large (8 cores)
And it seems that failing tests are affected with this. Some nodes are not getting enough CPU.

@lrubasze
Copy link
Copy Markdown
Contributor

lrubasze commented Jun 27, 2025

additionally full_node_warp_sync fails because of missing method GetParachainInfo_parachain_id

  21   │ 2025-06-27 11:28:24.574  INFO tokio-runtime-worker prometheus: [spiffy-disgust-2437] 〽️ Prometheus exporter started at 127.0.0.1:46145
  22   │ 2025-06-27 11:28:24.574  INFO                 main sc_rpc_server: [spiffy-disgust-2437] Running JSON-RPC server: addr=127.0.0.1:9945,[::1]:9945
  23   │ 2025-06-27 11:28:24.582  INFO                 main cumulus_test_service: [Parachain] Using inprocess node.
  24   │ 2025-06-27 11:28:24.583 ERROR tokio-runtime-worker sc_service::task_manager: [Parachain] Essential task `txpool-background` failed. Shutting down service.
  25   │ 2025-06-27 11:28:24.584 ERROR tokio-runtime-worker sc_service::task_manager: [spiffy-disgust-2437] Essential task `txpool-background` failed. Shutting down service.
  26   │ thread 'main' panicked at cumulus/test/service/src/main.rs:146:18:
  27   │ could not create Cumulus test service: Application(Application(Execution(Other("Exported method GetParachainInfo_parachain_id is not found"))))
  28   │ note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

This is related to this PR #8651

I will disable this test and create an issue to fix it.

@lrubasze
Copy link
Copy Markdown
Contributor

Failing zombienet-cumulus-0007-full_node_warp_sync issue: #9018

@lrubasze lrubasze requested review from a team as code owners June 27, 2025 13:43
@skunert
Copy link
Copy Markdown
Contributor Author

skunert commented Jun 27, 2025

Locally everything looking good, we can merge from my side!

@lrubasze lrubasze merged commit cd65182 into ci/zombienet-cumulus-tests-zombienet-sdk Jun 27, 2025
176 of 177 checks passed
@lrubasze lrubasze deleted the skunert/simplify-pov-recovery branch June 27, 2025 14:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

R0-no-crate-publish-required The change does not require any crates to be re-published. T10-tests This PR/Issue is related to tests. T18-zombienet_tests Trigger zombienet CI tests.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants