Skip to content

Refactor E2E suites to allow AHM integration#384

Merged
rockbmb merged 20 commits intomasterfrom
block-provider-refactor
Sep 10, 2025
Merged

Refactor E2E suites to allow AHM integration#384
rockbmb merged 20 commits intomasterfrom
block-provider-refactor

Conversation

@rockbmb
Copy link
Copy Markdown
Collaborator

@rockbmb rockbmb commented Sep 1, 2025

Closes #353, closes #382, closes #258.

Regarding #353

In order to keep a single test suite that can, with minimal changes, be run both on

  • live relay chain runtimes, and also
  • zombie-bite forks of post-migration Asset Hubs (used for testing)

some changes are necessary, most importantly to the scheduler E2E test suite.
This pallet, in a post-migration Asset Hub runtime setting, will use a non-local block provider, which changes some of the expectations that PET E2E tests have; see this PR for an example change.

Regarding #382

Furthermore, this PR also changes some dev accounts used in testing to fresh accounts with no state - Westend and Paseo runs of PET had false positives due to unclean states for Alice, Bob, etc.

Regarding #258

With the changes required by #353, the scheduler E2E test suite now has tests that are expected to run on relay chains, parachains with local block providers, and parachains with non-local block providers.
This entailed several .match() pattern matches to determine the correct offset at which to schedule tasks.

Documentation on the module and each test has been improved to clarify the new branching logic of these tests.


  • governance
  • multisig
  • nomination pools
  • staking
  • vesting
  • scheduler
  • proxy

Also, create new `testAccounts` object.
To run these tests in post-migration Asset Hubs, the scheduling technique
used to coerce execution of extrinsics with origins other than signed
must change to allow use of non-local block providers.

Also, when running these tests in testnets, some of the default dev
accounts have dirty states, so this commit begins the move to fresh
accounts in E2E.

Lastly, in some locations `scheduleInlineCallWithOrigin` was used without
an `await` before it, which led to occasionally wrong behavior.
@rockbmb rockbmb added this to the Kusama Migration milestone Sep 1, 2025
@rockbmb rockbmb self-assigned this Sep 1, 2025
@rockbmb rockbmb added enhancement New feature or request e2e tests Related to end-to-end tests refactor Changes to already-existing code. labels Sep 1, 2025
Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes are a significant and valuable refactoring to enable E2E tests on Asset Hub runtimes, primarily by abstracting away relay vs. para chain differences and using fresh accounts to prevent state conflicts. The addition of await to async calls fixes potential race conditions. However, there are several opportunities to improve clarity and robustness in the staking tests.

Comment thread packages/shared/src/staking.ts Outdated
Comment thread packages/shared/src/staking.ts
Comment thread packages/shared/src/staking.ts
@xlc
Copy link
Copy Markdown
Member

xlc commented Sep 2, 2025

can you merge master
and feel free to resolve/ignore AI review comments. but sometimes they are useful.

@rockbmb
Copy link
Copy Markdown
Collaborator Author

rockbmb commented Sep 7, 2025

CI is failing due to the introduction of a paseo package for local testing.

I will remove this before completing this PR.

* they now use the new `testAccounts` (snapshots (partially) updated,
* pass new `TestConfig` parameter to E2E test trees
* update proxy E2E tests to use new block provider (80% done)
@rockbmb
Copy link
Copy Markdown
Collaborator Author

rockbmb commented Sep 9, 2025

Kusama scheduler E2E tests are failing.

The behavior of scheduler.incompleteSince has changed on Kusama; if related to #397, I create a PR to fix tests/snapshots and then merge master into this branch.

@rockbmb rockbmb mentioned this pull request Sep 9, 2025
@rockbmb rockbmb requested a review from xlc September 9, 2025 21:04
@rockbmb
Copy link
Copy Markdown
Collaborator Author

rockbmb commented Sep 9, 2025

@xlc I still need to update scheduler tests for Kusama, but this is ready to be looked at.

In order to update proxy E2E tests for use with post-AHM parachains, I had to update them in the same manner you did with #399.
Polkadot relay proxy E2E tests are still failing for me, but after a fix, #398 can be closed in favor of this, if you're alright with that.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Sep 9, 2025

The changes introduce a new test configuration to support different block providers, which is a great improvement for testing across different chain types. However, I found one potential issue in the staking tests where a performance optimization is now only applied to local block providers, which might affect tests on other chains.


Review Suggestions

packages/shared/src/staking.ts:1948-1963

The logic to remove storage prefixes for ParasDisputes, Dmp, Staking, and Session is now only executed for 'Local' block providers. This might cause performance issues or flakiness in tests for non-local block providers, such as the Asset Hubs targeted by this PR. If this optimization is still needed, it should be applied to all block providers or have a clear justification for why it's not needed for non-local ones.

@rockbmb
Copy link
Copy Markdown
Collaborator Author

rockbmb commented Sep 10, 2025

The two failing tests belong to Kusama's scheduler E2E test suite.

They are:

  1. test that schedules an excessively overweight task, which should not be executed
    • /**
      * Test the process of
      *
      * 1. creating a call requiring a `Root` origin: an update to total issuance
      * 2. artificially manipulating that call's weight to the per-block weight limit allotted to scheduled calls
      * 3. scheduling the call
      * 4. checking that the call was not executed
      * 5. checking that it remains in the agenda for the original block it was scheduled in
      * @param chain
      *
      */
      export async function scheduledOverweightCallFails<
  2. test that schedules two differently-prioritized tasks, heavy tasks in the same block, and observes that priority is respected
    • /**
      * Test the process of
      *
      * 1. creating a call requiring a `Root` origin: an update to total issuance
      * 2. artificially manipulating that call's weight to the per-block weight limit allotted to scheduled calls
      * 3. scheduling the call
      * 4. checking that the call was not executed
      * 5. checking that it remains in the agenda for the original block it was scheduled in
      * @param chain
      *
      */
      export async function scheduledOverweightCallFails<

The first one is failing because on Kusama, scheduler.incompleteSince remains set after attempting to execute the overweight task, as expected.
The second fails because even after executing the lower-priority task, scheduler.incompleteSince remains set - this is not what should happen.

I will push a commit to allow different behavior on Kusama for now; investigating this is outside the scope of this PR, which has grown large enough.

Without it, the unapplied slashing test times out, because building
the era-change block takes some time to complete will all of the era's
stakers.
@rockbmb rockbmb enabled auto-merge (squash) September 10, 2025 14:00
@rockbmb rockbmb merged commit d5e0111 into master Sep 10, 2025
125 of 149 checks passed
@rockbmb rockbmb deleted the block-provider-refactor branch September 10, 2025 17:00
rockbmb added a commit that referenced this pull request Sep 12, 2025
PR #384 required some additional fixes that auto-merge could not perform.
rockbmb added a commit that referenced this pull request Sep 27, 2025
* Init balances e2e tests (wip)

* Rework initial tests for clarity

Most of them were written with recourse to Cursor, and were just not
very useful.

* Refactor `transfer_allow_death` test

It scrutinizes the extrinsic's events and their inner data.

* Recreate tests for all Polkadot/Kusama relay/SPs

* Rework networks that run `transferAllowDeath` test

Now, at the calling site of the test tree creation function, chains can
signal whether their ED is too low to run the basic `transferAllowDeath`
test.

* Scrutinize more events in `transferAllowDeath` test

* Test non-destructive `transfer_allow_death`

* Test `force_transfer` with account reaping

* Test to `force_transfer` reaping

* Test `transfer_allow_death` below ED

* Test `force_transfer` below ED

* Test overtransferring account

* Update comments and snapshots

* Test `force_transfer` with insufficient funds

* Test self `transfer_allow_death` of near-entire balance

* Update snapshots

* Update accounts tests after merge from `master`

PR #384 required some additional fixes that auto-merge could not perform.

* Update snapshots

* Fund new test account on Bridge Hub

* Revert change to Polkadot AH multisig suite name

* Create test to liquidity restrictions bug

See paritytech/polkadot-sdk#9560 and
paritytech/polkadot-sdk#8108.

TL;DR proxy and multisig pallets are using the old `Currency` trait
and not the new fungible traits, it is possible for an operation that
is permissible to an account at a given point in time and state to
fail.

See #401

* Test liquidity restrictions when creating proxy

* Parametrize deposit-requiring action liquidity restriction test

The `balances.LiquidityRestrictions` error can occur is different
contexts, as multiple pallets still use the old `Currency::reserve`
operation.

This commit parametrizes the test to allow for different kinds of
actions to be tested, not just proxy addition.

* Extend liquidity restriction tests to cover more cases

The test now combinatorially covers several cases:
1. Reserve action can vary between staking bond and nomination pool creation
2. Lock action is always vested transfer (for now)
3. Deposit action can vary between proxy creation, referendum submission
   and staking bond

* Test manual reserve/lock in liquidity restriction tests

This adds the possibility of manually set reserve/locks to the reserve/
lock actions used to combinatorially generate tests to liquidity
restrictions.

This is needed because some chains have no vesting or staking, but the
behavior should still be tested.

* Fix manual reserve/lock actions in liq. restriction tests

Fees were not being accounted for correctly when performing checks
on free/frozen balances.

* Update snapshots and accounts E2E test tree call sites

* Filter vesting as locking action on Asset Hubs

... while the AHM is pending - vesting operations are filtered, and
thus cannot be used.

* Add more comments to liquidity restriction test

Especially the action interfaces.

* Add more comments to liquidity restriction tests

* Remove leftover debug code from self-transfer test

* Fix Asset Hub test suites' use of relay chain

An invalid argument was being passed to the `scheduleInlineCallWithOrigin`
function: the base's chains block provider, instead of the relay chain's,
which is always `Local`.

* Snapshot skipped liquidity restriction tests

... instead of logging a message. Logs add some noise to CI test output.

* Apply lint fixes

* Update block numbers

* Change lock identifier used

* Remove unstable `Transfer` event from snapshots

* Update snapshots

* Simplify liquidity restriction tests

The `Balance.locks` storage is no longer modified - Polkadot Relay is
failing for an undetermined reason.

* Use nonces when setting storage

* Test `transfer_allow_death` with reserve

* Add test to `transfer_all` with reserve

* Add `force_transfer` with reserve test

* Add origin check tests for gated extrinsics

* Add tests to transfer functions with insufficient funds

* Test `transfer_all` with `keepAlive = true`

* Correct previous transfer all test snapshot

* Test `transfer_all` with `keep_alive = false`

* Correct snapshots of accounts tests

* Test `transfer_keep_alive` with source left below ED

* Split `transfer_keep_alive` tests

... into two separate tests:
1. one for chains with low ED (below a typical transfer fee)
2. one for chains with normal ED

* Test `force_adjust_total_issuance` with zero delta

* Test total issuance changes

* Test `force_unreserve` no-op cases

* Update accounts E2E test snapshots

* Update liquidity restriction test snapshots for AHs

* Test forceful unreservation of funds

* Add more self-transfer tests

* Add self-transfer test to `force_transfer`

* Add tests to `force_set_balance`

* Refactor `burn` tests into low/normal ED tests

* Update snapshots with new `burn` tests

* Test burning from account with consumer reference

* Add test to burning of entire balance (or more than it)

* Update snapshots to remove obsolete data

* Improve documentation and `README`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

e2e tests Related to end-to-end tests enhancement New feature or request refactor Changes to already-existing code.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use different dev accounts by default Abstract block number provider from E2E tests Improve scheduler E2E test suite

2 participants