Add optional auto-rebag within on-idle by nkpar · Pull Request #8684 · paritytech/polkadot-sdk

nkpar · 2025-05-28T01:36:09Z

Implemented an on_idle hook for automatic rebagging of accounts, enabling incremental correction of account positions within the bags-list during the idle phase of block execution. Introduced new storages and configurable constants to support this functionality, along with improvements in logging and internal helpers.

Implemented an `on_idle` hook for automatic rebagging of accounts, enabling incremental correction of account positions within the bags-list during the idle phase of block execution. Introduced new storages and configurable constants to support this functionality, along with improvements in logging and internal helpers.

Make it work on kitchensink-runtime

…bag-within-on-idle

substrate/frame/bags-list/src/lib.rs

Refactored rebagging logic into `rebag_internal` to simplify code and improve readability. Introduced the `AutoRebagPerBlock` type, including its integration into relevant runtimes.

…-idle' of https://github.com/paritytech/polkadot-sdk into 8403-pallet-bags-list-add-optional-auto-rebag-within-on-idle

Clarify the behavior of the `NextNodeAutoRebagging` pointer to indicate the next node for rebagging rather than the last auto-rebagged node. Adjust comments and logs for improved accuracy and readability, ensuring consistent and precise descriptions.

nkpar · 2025-05-28T17:05:37Z

/cmd --help

github-actions · 2025-05-28T17:06:09Z

Command help:

usage: /cmd  [--help] [--quiet] [--clean] [--image IMAGE]
             {bench,bench-omni,fmt,update-ui,prdoc} ...

A command runner for polkadot-sdk repo

positional arguments:
  {bench,bench-omni,fmt,update-ui,prdoc}
                        a command to run
    bench (bench-omni)  Runs benchmarks (frame omni bencher)
    fmt                 Formats code (cargo +nightly-VERSION fmt) and configs
                        (taplo format)
    update-ui           Updates UI tests
    prdoc               Generates PR documentation

options:
  --help                help for help if you need some help
  --quiet               Won't print start/end/failed messages in PR
  --clean               Clean up the previous bot's & author's comments in PR
  --image IMAGE         Override docker image '--image
                        docker.io/paritytech/ci-unified:latest'

### Command 'bench'
usage: /cmd bench [-h] [--quiet] [--clean] [--image IMAGE]
                  [--runtime [{dev,westend,rococo,asset-hub-westend,asset-hub-rococo,bridge-hub-rococo,bridge-hub-westend,collectives-westend,coretime-rococo,coretime-westend,glutton-westend,people-rococo,people-westend} ...]]
                  [--pallet [PALLET ...]] [--fail-fast]

options:
  -h, --help            show this help message and exit
  --quiet               Won't print start/end/failed messages in PR
  --clean               Clean up the previous bot's & author's comments in PR
  --image IMAGE         Override docker image '--image
                        docker.io/paritytech/ci-unified:latest'
  --runtime [{dev,westend,rococo,asset-hub-westend,asset-hub-rococo,bridge-hub-rococo,bridge-hub-westend,collectives-westend,coretime-rococo,coretime-westend,glutton-westend,people-rococo,people-westend} ...]
                        Runtime(s) space separated
  --pallet [PALLET ...]
                        Pallet(s) space separated
  --fail-fast           Fail fast on first failed benchmark

**Examples**:
 Runs all benchmarks 
 /cmd bench

 Runs benchmarks for pallet_balances and pallet_multisig for all runtimes which have these pallets. **--quiet** makes it to output nothing to PR but reactions
 /cmd bench --pallet pallet_balances pallet_xcm_benchmarks::generic --quiet
 
 Runs bench for all pallets for westend runtime and fails fast on first failed benchmark
 /cmd bench --runtime westend --fail-fast
 
 Does not output anything and cleans up the previous bot's & author command triggering comments in PR 
 /cmd bench --runtime westend rococo --pallet pallet_balances pallet_multisig --quiet --clean


### Command 'bench-omni'
usage: /cmd bench [-h] [--quiet] [--clean] [--image IMAGE]
                  [--runtime [{dev,westend,rococo,asset-hub-westend,asset-hub-rococo,bridge-hub-rococo,bridge-hub-westend,collectives-westend,coretime-rococo,coretime-westend,glutton-westend,people-rococo,people-westend} ...]]
                  [--pallet [PALLET ...]] [--fail-fast]

options:
  -h, --help            show this help message and exit
  --quiet               Won't print start/end/failed messages in PR
  --clean               Clean up the previous bot's & author's comments in PR
  --image IMAGE         Override docker image '--image
                        docker.io/paritytech/ci-unified:latest'
  --runtime [{dev,westend,rococo,asset-hub-westend,asset-hub-rococo,bridge-hub-rococo,bridge-hub-westend,collectives-westend,coretime-rococo,coretime-westend,glutton-westend,people-rococo,people-westend} ...]
                        Runtime(s) space separated
  --pallet [PALLET ...]
                        Pallet(s) space separated
  --fail-fast           Fail fast on first failed benchmark

**Examples**:
 Runs all benchmarks 
 /cmd bench

 Runs benchmarks for pallet_balances and pallet_multisig for all runtimes which have these pallets. **--quiet** makes it to output nothing to PR but reactions
 /cmd bench --pallet pallet_balances pallet_xcm_benchmarks::generic --quiet
 
 Runs bench for all pallets for westend runtime and fails fast on first failed benchmark
 /cmd bench --runtime westend --fail-fast
 
 Does not output anything and cleans up the previous bot's & author command triggering comments in PR 
 /cmd bench --runtime westend rococo --pallet pallet_balances pallet_multisig --quiet --clean


### Command 'fmt'
usage: /cmd fmt [-h] [--quiet] [--clean] [--image IMAGE]

options:
  -h, --help     show this help message and exit
  --quiet        Won't print start/end/failed messages in PR
  --clean        Clean up the previous bot's & author's comments in PR
  --image IMAGE  Override docker image '--image docker.io/paritytech/ci-
                 unified:latest'


### Command 'update-ui'
usage: /cmd update-ui [-h] [--quiet] [--clean] [--image IMAGE]

options:
  -h, --help     show this help message and exit
  --quiet        Won't print start/end/failed messages in PR
  --clean        Clean up the previous bot's & author's comments in PR
  --image IMAGE  Override docker image '--image docker.io/paritytech/ci-
                 unified:latest'


### Command 'prdoc'
usage: /cmd prdoc [-h] [--pr PR]
                  [--audience [{runtime_dev,runtime_user,node_dev,node_operator,todo} ...]]
                  [--bump {patch,minor,major,silent,ignore,none}] [--force]

options:
  -h, --help            show this help message and exit
  --pr PR               The PR number to generate the PrDoc for.
  --audience [{runtime_dev,runtime_user,node_dev,node_operator,todo} ...]
                        The audience of whom the changes may concern. Example:
                        --audience runtime_dev node_dev
  --bump {patch,minor,major,silent,ignore,none}
                        A default bump level for all crates. Example: --bump
                        patch
  --force               Whether to overwrite any existing PrDoc.

nkpar · 2025-05-28T17:06:49Z

/cmd fmt

nkpar · 2025-05-28T17:15:37Z

/cmd prdoc --audience runtime_dev --bump minor

…time_dev --bump minor'

Make tests work

Improve auto-rebagging with error tracking and logging. Added detailed logging for weight limits, successful and failed rebags, and unexpected cursor states. Enhanced robustness by handling pallet lock states and iterator failures gracefully.

nkpar · 2025-05-28T21:52:21Z

/cmd fmt

Ank4n

Looking great — I just want to do one more pass before approving.

substrate/frame/bags-list/src/lib.rs

…bag-within-on-idle

substrate/frame/bags-list/src/lib.rs

kianenigma

Looks very good so far 👍 mainly missing some unit tests

substrate/frame/bags-list/src/lib.rs

Removed the requirement to pass the score provider function as a parameter to `rebag_internal`.

Use `defensive!` macro for logging in auto-rebag errors.

nkpar · 2025-07-01T15:22:28Z

/cmd bench --pallet pallet_bags_list

github-actions · 2025-07-01T15:23:26Z

Command "bench --pallet pallet_bags_list" has started 🚀 See logs here

paritytech-workflow-stopper · 2025-07-01T15:56:38Z

All GitHub workflows were cancelled due to failure one of the required jobs.
Failed workflow url: https://github.com/paritytech/polkadot-sdk/actions/runs/16003434560
Failed job name: test-linux-stable-runtime-benchmarks

…t_bags_list'

github-actions · 2025-07-01T16:17:38Z

Command "bench --pallet pallet_bags_list" has finished ✅ See logs here

Details

Subweight results:

File	Extrinsic	Old	New	Change [%]
cumulus/parachains/runtimes/assets/asset-hub-westend/src/weights/pallet_bags_list.rs	put_in_front_of	917.43us	1.08ms	+17.20
cumulus/parachains/runtimes/assets/asset-hub-westend/src/weights/pallet_bags_list.rs	rebag_non_terminal	737.83us	862.27us	+16.87
cumulus/parachains/runtimes/assets/asset-hub-westend/src/weights/pallet_bags_list.rs	rebag_terminal	735.59us	857.28us	+16.54
substrate/frame/bags-list/src/weights.rs	rebag_non_terminal	729.72us	773.70us	+6.03
substrate/frame/bags-list/src/weights.rs	rebag_terminal	728.18us	770.63us	+5.83
substrate/frame/staking-async/runtimes/parachain/src/weights/pallet_bags_list.rs	on_idle		8.07ms	Added
substrate/frame/bags-list/src/weights.rs	on_idle		8.17ms	Added
polkadot/runtime/westend/src/weights/pallet_bags_list.rs	on_idle		8.17ms	Added
cumulus/parachains/runtimes/assets/asset-hub-westend/src/weights/pallet_bags_list.rs	on_idle		8.54ms	Added

Command output:

✅ Successful benchmarks of runtimes/pallets:
-- dev: ['pallet_bags_list']
-- westend: ['pallet_bags_list']
-- asset-hub-westend: ['pallet_bags_list']

Implemented an `on_idle` hook for automatic rebagging of accounts, enabling incremental correction of account positions within the bags-list during the idle phase of block execution. Introduced new storages and configurable constants to support this functionality, along with improvements in logging and internal helpers. --------- Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Ankan <10196091+Ank4n@users.noreply.github.com> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

…9166) quick follow-up to #8684, ensuring all blocks don't have x events when the feature is enabled (as it is now in WAH) --------- Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* master: (91 commits) Add extra information to the harmless error logs during validate_transaction (#9047) `sp-tracing`: Remove `test-utils` feature (#9063) add try-state check for staking roles -- staker cannot be nominator a… (#9034) net/discovery: File persistence for `AddrCache` (#8839) dispute-coordinator: handle race with offchain disabling (#9050) Align parameters for `EventEmitter::emit_sent_event` (#9057) Fetch parent block `api_version` (#9059) [XCM Precompile] Rename functions and improve docs in the Solidity interface (#9023) Cleanup and improvements for `ControlledValidatorIndices` (#8896) reenable 0001-parachains-pvf (#9046) Add optional auto-rebag within on-idle (#8684) Fix flaxy 0003-block-building-warp-sync test - one more approach (#8974) [Staking] [AHM] Fixes insufficient slashing of nominators (and some other small issues). (#8937) chore: Bump bounded-collections dep (#9004) XCMP and DMP improvements (#8860) EPMB/unsigned: fixed multi-page winner computation (#8987) Always send full parent header, not only hash, part of collation response (#8939) revive: Precompiles should return dummy code when queried (#9001) Fix confusing log messages in network protocol behaviour (#8819) Fix pallet_migrations benchmark when FailedMigrationHandler emits events (#8694) ...

@kianenigma

This brings in `unstable2507` Polkadot SDK, and integrates new features. Integrated breaking changes to be verified by the original authors: - [ ] paritytech/polkadot-sdk#7953 @kianenigma @Ank4n acatangiu#13 - [x] paritytech/polkadot-sdk#8684 @nkpar - [x] paritytech/polkadot-sdk#8693 @tiagobndr @franciscoaguirre - [x] paritytech/polkadot-sdk#9137 @franciscoaguirre fixes #837 --------- Co-authored-by: Branislav Kontur <bkontur@gmail.com> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Co-authored-by: kianenigma <kian@parity.io> Co-authored-by: Christian Langenbacher <clangenb+gh@protonmail.ch>

Implemented an `on_idle` hook for automatic rebagging of accounts, enabling incremental correction of account positions within the bags-list during the idle phase of block execution. Introduced new storages and configurable constants to support this functionality, along with improvements in logging and internal helpers. --------- Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Ankan <10196091+Ank4n@users.noreply.github.com> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

…9166) quick follow-up to #8684, ensuring all blocks don't have x events when the feature is enabled (as it is now in WAH) --------- Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>

nkpar linked an issue May 28, 2025 that may be closed by this pull request

pallet-bags-list: Add optional auto-rebag within on-idle #8403

Closed

kitchensink-runtime adoption

ca81567

Make it work on kitchensink-runtime

nkpar added the T2-pallets This PR/Issue is related to a particular pallet. label May 28, 2025

nkpar changed the title ~~WIP~~ Add optional auto-rebag within on-idle May 28, 2025

Merge branch 'master' into 8403-pallet-bags-list-add-optional-auto-re…

6b6e572

…bag-within-on-idle

kianenigma reviewed May 28, 2025

View reviewed changes

substrate/frame/bags-list/src/lib.rs Outdated Show resolved Hide resolved

Ank4n self-requested a review May 28, 2025 14:37

nkpar added 3 commits May 28, 2025 16:29

Review comments and runtimes support

6977e70

Refactored rebagging logic into `rebag_internal` to simplify code and improve readability. Introduced the `AutoRebagPerBlock` type, including its integration into relevant runtimes.

Merge branch '8403-pallet-bags-list-add-optional-auto-rebag-within-on…

40fe3ef

…-idle' of https://github.com/paritytech/polkadot-sdk into 8403-pallet-bags-list-add-optional-auto-rebag-within-on-idle

Update from github-actions[bot] running command 'fmt'

094da4d

github-actions bot and others added 4 commits May 28, 2025 17:17

Update from github-actions[bot] running command 'prdoc --audience run…

ec01134

…time_dev --bump minor'

Update pr_8684.prdoc

c88dc5b

Update mock.rs

cadf8c8

Make tests work

Update lib.rs

d118b26

Improve auto-rebagging with error tracking and logging. Added detailed logging for weight limits, successful and failed rebags, and unexpected cursor states. Enhanced robustness by handling pallet lock states and iterator failures gracefully.

Update from github-actions[bot] running command 'fmt'

a552dae

Ank4n reviewed May 29, 2025

View reviewed changes

substrate/frame/bags-list/src/lib.rs Outdated Show resolved Hide resolved

substrate/frame/bags-list/src/lib.rs Outdated Show resolved Hide resolved

Merge branch 'master' into 8403-pallet-bags-list-add-optional-auto-re…

c2b6cc7

…bag-within-on-idle

kianenigma reviewed May 29, 2025

View reviewed changes

substrate/frame/bags-list/src/lib.rs Outdated Show resolved Hide resolved

kianenigma reviewed May 29, 2025

View reviewed changes

substrate/frame/bags-list/src/lib.rs Outdated Show resolved Hide resolved

kianenigma reviewed May 29, 2025

View reviewed changes

substrate/frame/bags-list/src/lib.rs Show resolved Hide resolved

nkpar added 2 commits May 29, 2025 11:42

Refactor rebag_internal and improve mocs readability

4f86865

Removed the requirement to pass the score provider function as a parameter to `rebag_internal`.

Update lib.rs

296fe0c

Use `defensive!` macro for logging in auto-rebag errors.

Update pr_8684.prdoc

b8f0f10

paritytech deleted a comment from github-actions bot Jul 1, 2025

Update from github-actions[bot] running command 'bench --pallet palle…

50e930b

…t_bags_list'

nkpar enabled auto-merge July 1, 2025 16:47

nkpar added this pull request to the merge queue Jul 1, 2025

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jul 1, 2025

nkpar added this pull request to the merge queue Jul 1, 2025

Merged via the queue into master with commit 6824a67 Jul 1, 2025
241 checks passed

nkpar deleted the 8403-pallet-bags-list-add-optional-auto-rebag-within-on-idle branch July 1, 2025 18:08

kianenigma mentioned this pull request Jul 10, 2025

pallet-bags-list: Emit ScoreUpdated event only if it has changed #9166

Merged

rockbmb mentioned this pull request Jul 23, 2025

Bump PET submodule paritytech/ahm-dryrun#119

Closed

acatangiu mentioned this pull request Aug 12, 2025

Upgrade to Polkadot-SDK unstable2507 polkadot-fellows/runtimes#849

Merged

4 tasks

pgherveou added this to [preview] release tracker Mar 20, 2026

github-project-automation bot moved this to Done in [preview] release tracker Mar 20, 2026

pgherveou removed this from [preview] release tracker Mar 20, 2026

pgherveou added this to [preview] release tracker Mar 23, 2026

github-project-automation bot moved this to Done in [preview] release tracker Mar 23, 2026

Conversation

nkpar commented May 28, 2025

Uh oh!

Uh oh!

nkpar commented May 28, 2025

Uh oh!

github-actions bot commented May 28, 2025

Uh oh!

nkpar commented May 28, 2025

Uh oh!

nkpar commented May 28, 2025

Uh oh!

nkpar commented May 28, 2025

Uh oh!

Ank4n left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kianenigma left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nkpar commented Jul 1, 2025

Uh oh!

github-actions bot commented Jul 1, 2025

Uh oh!

paritytech-workflow-stopper bot commented Jul 1, 2025

Uh oh!

github-actions bot commented Jul 1, 2025

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants