Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
The CI rework effectively groups Asset Hub tests to share Chopsticks instances, which is good for resource usage. However, running tests sequentially within these groups is inefficient compared to Vitest's native parallelism. There is also a potential edge case in the test filtering logic that could cause some tests to be skipped entirely.
rockbmb
changed the title
Test matrix changes reverted, but chopsticks still used as a cache. Instead of having a separate matrix for PAH/KAH, each job in the original matrix will build its own chopsticks fork from a DB created in an early step (only applies to PAH/KAH tests).
Previously, it was `system.applyAuthorizedUpgrade`, but this leads to the occasional spurious error when PET's CI workflows are scheduled to run at/near a runtime upgrade. Now, `system.setHeapPages` is used: it requires a root origin and will this fail, but recall that the purpose of the test is to check proxy call filters work, and that call filters are enacted *before* origin checks, which only occur within the extrinsic's execution.
xlc
approved these changes
Feb 1, 2026
fellowship-merge-bot Bot
pushed a commit
to polkadot-fellows/runtimes
that referenced
this pull request
Feb 19, 2026
Closes #1010 . This PR will rework the PET job in `.github/workflows/test.yml`. - [x] use a test matrix with individual retry/timeout configurations rather than just `yarn test` to avoid restarting the whole suite on a single suite's failure - [x] To also avoid RPC timeouts due to throttling/stalling by test network's production RPC endpoints, consider using https://github.com/AcalaNetwork/subway or Chopsticks (as in open-web3-stack/polkadot-ecosystem-tests#504) - [x] Filter end-to-end test suites from PET to be used in finalized workflow - This repository is only concerned with the runtimes that are under the Polkadot Fellowship's purview, while PET contains end-to-end tests for other ecosystme chains; as such, it makes sense to filter what is included in CI to further reduce load. - [x] Re-enable parts of `test.yml` that were disabled to test PET changes in isolation # Review notes Main changes: * new folder in `.github/subway-configs`; it contains [Subway](https://github.com/AcalaNetwork/subway) configs for the RPC proxies that will be created during CI: one for each Polkadot/Kusama relay + system parachain . * PET reenabled in `.github/workflows/test.yml`, and completely refactored - it uses a matrix strategy with two jobs, one for each network: Polkadot and Kusama ## Details Each of the above P/K jobs: - Fetches Rust toolchain + Subway, builds its binary, and starts a Subway proxy for each of that network's relay/SP - Polkadot: 1 proxy for relay + 5 proxies for SPs - Kusama: 1 proxy for relay + 4 proxies for SPs - Downloads WASM artifacts from a previous step of the CI pipeline - Overrides PET's `.env` with said runtimes + proxies' endpoints - That means PET will run using runtimes built against the code of whichever PR triggered CI, and cache RPC responses, which are shared between all chains' tests - Gets `yarn`, checks out PET repo, builds it - Runs PET's known block number command from within it, to use the latest state in combination with built WASM runtimes - Runs all end-to-end test suites for that network's chains - only runs tests related to relay + SPs; ecosystem chains are skipped, as this repo's purview is only for Fellowship-maintained runtimes - Cleans up Subway proxies at the end In particular, the critical change: ```bash yarn test -u $TEST_FILES --pool=threads --maxWorkers=3 --retry=3 # `$TEST_FILES` contains all E2E modules for the network: relay + SPs included # 1. `-u` updates snapshots so that they don't fail spuriously e.g. proof size/ref time changes # 2. `--pool=threads` runs test files from `$TEST_FILES` in parallel using worker threads. Default # is `--pool=forks`, which uses child processes - heavier. # 3. `--maxWorkers=3` combined with the above means at most 3 test files running concurrently, each in its # own thread # 4. `--retry=3` attempts a failing test 3 times. This operates at the level of individual tests, not modules, # or the whole test suite; one failure will *not* require a full rerun. ``` --- - [x] Does not require a CHANGELOG entry
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
WIP, still need to create an issue describing what this solves.Closes #506 .
TL;DR a Chopsticks DB is created in CI and given to all jobs to cache some data and avoid RPC timeouts..