pallet_revive: Replace adhoc pre-compiles with pre-compile framework by athei · Pull Request #8262 · paritytech/polkadot-sdk

athei · 2025-04-16T15:58:54Z

This PRs adds the ability to define custom pre-compiles from outside the pallet. Before, all pre-compiles were hard coded as part of the pallet.

Design

Adding a pre-compile is as easy as implementing the new Precompile trait on any type. It can be added to the pallet by passing it into Config::Precompiles. This config know excepts a tuple of multiple types that each implement Precompile.
Each pre-compile has to implement Solidity ABI. Meaning its inputs and outputs are encoded according to Eth ABI. This makes writing a pre-compile much simpler since it doesn't have to implement its own decoding logic. More importantly: It makes it trivial to consume the API from a Solidity contract.
We constrain the address space of pre-compiles to a safe range so that they cannot accidentally match a wide range creating a collision with real contracts.
We check that pre-compile address ranges do not overlap at compile time.
Pre-compiles behave exactly as a normal contract. They exist as frames on the call stack and the environment they observe is their own (not the one of the calling contract). They can also be delegate called which changes the semantics in the same way as for normal contracts: They observe the environment of the calling contract.
They can also be called by the origin without any other contract in-between.

Check the rustdocs of the precompile module on how to write a pre-compile.

Changes

A new module precompiles is added that contains the framework to write pre-compiles. It also contains the sub module builtin that contains hard coded pre-compiles which exist Ethereum.
The pure_precompiles module was deleted since all its pre-compiles were ported over to the new framework and are now housed in builtin.
The CallSetup type is moved outside of the benchmarking module because it is also needed for testing code now. It is intended to be used for implementors outside of the crate to test the pre-compiles (in addition to benchmarking them).

Follow Ups

Fixes #6716

athei · 2025-04-16T16:02:28Z

/cmd fmt

athei · 2025-04-16T16:06:28Z

/cmd bench --runtime dev --pallet pallet_revive

github-actions · 2025-04-16T16:07:26Z

Command "bench --runtime dev --pallet pallet_revive" has started 🚀 See logs here

…--pallet pallet_revive'

github-actions · 2025-04-16T16:44:25Z

Command "bench --runtime dev --pallet pallet_revive" has finished ✅ See logs here

Details

Subweight results:

File	Extrinsic	Old	New	Change [%]
substrate/frame/revive/src/weights.rs	bn128_add	15.82us	743.36us	+4598.85
substrate/frame/revive/src/weights.rs	blake2f	27.67us	751.38us	+2615.59
substrate/frame/revive/src/weights.rs	identity	39.19us	766.60us	+1856.16
substrate/frame/revive/src/weights.rs	ecdsa_recover	45.17us	772.35us	+1609.75
substrate/frame/revive/src/weights.rs	sha2_256	304.01us	1.08ms	+254.97
substrate/frame/revive/src/weights.rs	ripemd_160	990.89us	1.77ms	+78.72
substrate/frame/revive/src/weights.rs	bn128_mul	1.02ms	1.75ms	+71.68
substrate/frame/revive/src/weights.rs	seal_copy_to_contract	62.81us	77.86us	+23.98
substrate/frame/revive/src/weights.rs	seal_return	63.03us	77.64us	+23.17
substrate/frame/revive/src/weights.rs	seal_to_account_id	31.68us	34.76us	+9.73
substrate/frame/revive/src/weights.rs	seal_contains_transient_storage	1.95us	1.83us	-5.83
substrate/frame/revive/src/weights.rs	seal_balance	5.17us	4.81us	-6.98
substrate/frame/revive/src/weights.rs	seal_weight_to_fee	1.60us	1.47us	-8.36
substrate/frame/revive/src/weights.rs	rollback_transient_storage	1.21us	1.11us	-8.41
substrate/frame/revive/src/weights.rs	instr	1.57ms	1.35ms	-13.66
substrate/frame/revive/src/weights.rs	seal_weight_left	736.00ns	610.00ns	-17.12
substrate/frame/revive/src/weights.rs	seal_caller_is_origin	347.00ns	232.00ns	-33.14
substrate/frame/revive/src/weights.rs	seal_gas_limit	482.00ns	306.00ns	-36.51
substrate/frame/revive/src/weights.rs	seal_gas_price	276.00ns	168.00ns	-39.13
substrate/frame/revive/src/weights.rs	seal_caller_is_root	299.00ns	174.00ns	-41.81
substrate/frame/revive/src/weights.rs	seal_own_code_hash	305.00ns	172.00ns	-43.61
substrate/frame/revive/src/weights.rs	seal_address	346.00ns	189.00ns	-45.38
substrate/frame/revive/src/weights.rs	seal_base_fee	292.00ns	158.00ns	-45.89
substrate/frame/revive/src/weights.rs	seal_minimum_balance	282.00ns	151.00ns	-46.45
substrate/frame/revive/src/weights.rs	seal_call_data_load	293.00ns	153.00ns	-47.78
substrate/frame/revive/src/weights.rs	seal_now	290.00ns	150.00ns	-48.28
substrate/frame/revive/src/weights.rs	seal_return_data_size	285.00ns	140.00ns	-50.88
substrate/frame/revive/src/weights.rs	seal_ref_time_left	276.00ns	135.00ns	-51.09
substrate/frame/revive/src/weights.rs	seal_value_transferred	292.00ns	140.00ns	-52.05
substrate/frame/revive/src/weights.rs	seal_call_data_size	320.00ns	141.00ns	-55.94
substrate/frame/revive/src/weights.rs	seal_block_number	308.00ns	135.00ns	-56.17
substrate/frame/revive/src/weights.rs	seal_caller	1.13us	217.00ns	-80.80
substrate/frame/revive/src/weights.rs	seal_origin	1.09us	182.00ns	-83.38
substrate/frame/revive/src/weights.rs	seal_call_precompile		486.83us	Added

Command output:

✅ Successful benchmarks of runtimes/pallets:
-- dev: ['pallet_revive']

athei · 2025-04-16T21:05:51Z

/cmd bench --runtime dev --pallet pallet_revive

github-actions · 2025-04-16T21:07:10Z

Command "bench --runtime dev --pallet pallet_revive" has started 🚀 See logs here

…--pallet pallet_revive'

github-actions · 2025-04-16T21:40:18Z

Command "bench --runtime dev --pallet pallet_revive" has finished ✅ See logs here

Details

Subweight results:

File	Extrinsic	Old	New	Change [%]
substrate/frame/revive/src/weights.rs	seal_return	63.03us	78.04us	+23.81
substrate/frame/revive/src/weights.rs	seal_copy_to_contract	62.81us	77.67us	+23.67
substrate/frame/revive/src/weights.rs	sha2_256	304.01us	341.36us	+12.29
substrate/frame/revive/src/weights.rs	seal_to_account_id	31.68us	34.67us	+9.46
substrate/frame/revive/src/weights.rs	bn128_add	15.82us	16.89us	+6.73
substrate/frame/revive/src/weights.rs	get_transient_storage_empty	1.48us	1.39us	-6.41
substrate/frame/revive/src/weights.rs	get_transient_storage_full	1.64us	1.53us	-6.42
substrate/frame/revive/src/weights.rs	seal_balance	5.17us	4.81us	-6.98
substrate/frame/revive/src/weights.rs	seal_contains_transient_storage	1.95us	1.80us	-7.36
substrate/frame/revive/src/weights.rs	seal_weight_to_fee	1.60us	1.48us	-7.80
substrate/frame/revive/src/weights.rs	rollback_transient_storage	1.21us	1.08us	-10.72
substrate/frame/revive/src/weights.rs	seal_weight_left	736.00ns	637.00ns	-13.45
substrate/frame/revive/src/weights.rs	instr	1.57ms	1.30ms	-17.18
substrate/frame/revive/src/weights.rs	seal_gas_limit	482.00ns	370.00ns	-23.24
substrate/frame/revive/src/weights.rs	seal_caller_is_root	299.00ns	203.00ns	-32.11
substrate/frame/revive/src/weights.rs	seal_caller_is_origin	347.00ns	228.00ns	-34.29
substrate/frame/revive/src/weights.rs	seal_ref_time_left	276.00ns	168.00ns	-39.13
substrate/frame/revive/src/weights.rs	seal_gas_price	276.00ns	163.00ns	-40.94
substrate/frame/revive/src/weights.rs	seal_address	346.00ns	204.00ns	-41.04
substrate/frame/revive/src/weights.rs	seal_value_transferred	292.00ns	170.00ns	-41.78
substrate/frame/revive/src/weights.rs	seal_own_code_hash	305.00ns	176.00ns	-42.30
substrate/frame/revive/src/weights.rs	seal_block_number	308.00ns	167.00ns	-45.78
substrate/frame/revive/src/weights.rs	seal_now	290.00ns	156.00ns	-46.21
substrate/frame/revive/src/weights.rs	seal_minimum_balance	282.00ns	151.00ns	-46.45
substrate/frame/revive/src/weights.rs	seal_base_fee	292.00ns	154.00ns	-47.26
substrate/frame/revive/src/weights.rs	seal_return_data_size	285.00ns	149.00ns	-47.72
substrate/frame/revive/src/weights.rs	seal_call_data_load	293.00ns	142.00ns	-51.54
substrate/frame/revive/src/weights.rs	seal_call_data_size	320.00ns	154.00ns	-51.88
substrate/frame/revive/src/weights.rs	seal_caller	1.13us	233.00ns	-79.38
substrate/frame/revive/src/weights.rs	seal_origin	1.09us	206.00ns	-81.19
substrate/frame/revive/src/weights.rs	seal_call_precompile		486.20us	Added

Command output:

✅ Successful benchmarks of runtimes/pallets:
-- dev: ['pallet_revive']

substrate/frame/revive/src/precompiles.rs

substrate/frame/revive/src/tests.rs

substrate/frame/revive/src/lib.rs

substrate/frame/revive/src/precompiles.rs

Co-authored-by: PG Herveou <pgherveou@gmail.com>

substrate/frame/revive/src/benchmarking.rs

substrate/frame/revive/src/exec.rs

substrate/frame/revive/src/precompiles.rs

pgherveou · 2025-04-24T09:32:28Z

You can also get rid of https://github.com/paritytech/polkadot-sdk/blob/f0c95b7ad8a41666f2fe51fa8ab2077b16a1617e/substrate/frame/revive/uapi/src/host.rs?plain=1#L459-L436

Co-authored-by: PG Herveou <pgherveou@gmail.com>

xermicus

Looks really good!

Just one thing: I tried the revive integration tests against this branch and there is a failing case passing against master: Delegate calls to unknown (non-existing) contract address revert with an out of gas error. I could not pin down the root cause quickly but to me this looks like a small regression?

substrate/frame/revive/src/exec.rs

substrate/frame/revive/src/precompiles.rs

substrate/frame/revive/src/precompiles/tests.rs

Co-authored-by: xermicus <cyrill@parity.io>

athei · 2025-04-25T16:44:13Z

Looks really good!

Just one thing: I tried the revive integration tests against this branch and there is a failing case passing against master: Delegate calls to unknown (non-existing) contract address revert with an out of gas error. I could not pin down the root cause quickly but to me this looks like a small regression?

Can you send me a link to the test code in question?

xermicus · 2025-04-25T19:49:16Z

https://github.com/paritytech/revive/blob/f6a412eef4d5aa9779b8499713e53c2b77a91ad3/crates/integration/contracts/DelegateCaller.sol#L31

I see the out of gas revert not only for the zero address but any arbitrary address I set there.

athei · 2025-04-26T09:44:37Z

I made a change that the gas for loading the code from storage is charged from the nested meter and not the parent meter. But in your test all remaining gas is passed. So it it should not affect this.

I see the out of gas revert not only for the zero address but any arbitrary address I set there.

Weird. As if delegate calls don't work at all. However, we have tests for delegate calls which continued to work unchanged. I have to investigate further. I will try to run this test against my dev node and add some logs.

xermicus · 2025-04-26T13:04:20Z

The arguments stay the same so I assume there must be something changed in the pallet.

Trace log from the PR branch:

2025-04-26T12:40:18.460783Z TRACE runtime::revive::strace: delegate_call(flags_and_callee: 196624, ref_time_limit: 18446744073709551615, proof_size_limit: 18446744073709551615, deposit_ptr: 4294836128, input_data: 17180000432, output_data: 18446180574000447504) = Err(TrapReason::SupervisorError(Module(ModuleError { index: 3, error: [3, 0, 0, 0], message: Some("OutOfGas") }))) gas_consumed: Weight { ref_time: 100000000000000, proof_size: 3221225472 }

On master:

2025-04-26T12:57:16.755145Z TRACE runtime::revive::strace: delegate_call(flags_and_callee: 196624, ref_time_limit: 18446744073709551615, proof_size_limit: 18446744073709551615, deposit_ptr: 4294836128, input_data: 17180000432, output_data: 18446180574000447504) = Ok(Success) gas_consumed: Weight { ref_time: 121866192, proof_size: 7557 }

The weight in that log corresponds to the sandbox runners default limit.. Or is the runners config wrong? To test against this branch I just change the ChainExtension type to Precompiles.

athei · 2025-04-26T18:27:16Z

The arguments stay the same so I assume there must be something changed in the pallet.

Agreed. It is a regression. Didn't had time to track it down, yet.

athei · 2025-04-28T15:10:13Z

So I think I found the bug. When we error out early when creating a new frame we never refund the nested gas into the parent meter. The bug was introduced earlier but the changes here added one more case where this can happen: This is because I moved the check if the delegated to contract exists after the spawning of the nested meter.

I did a bit of research: What we should do is that everything except the execution costs should be charged from the parent meter as this is what EVM does.

I will add a regression test and fix this.

paritytech-workflow-stopper · 2025-04-28T16:19:12Z

All GitHub workflows were cancelled due to failure one of the required jobs.
Failed workflow url: https://github.com/paritytech/polkadot-sdk/actions/runs/14712012332
Failed job name: test-linux-stable

xermicus · 2025-05-05T06:33:49Z

So I think I found the bug. When we error out early when creating a new frame we never refund the nested gas into the parent meter. The bug was introduced earlier but the changes here added one more case where this can happen: This is because I moved the check if the delegated to contract exists after the spawning of the nested meter.

I did a bit of research: What we should do is that everything except the execution costs should be charged from the parent meter as this is what EVM does.

I will add a regression test and fix this.

Awesome thanks for digging this up!

…8262) This PRs adds the ability to define custom pre-compiles from outside the pallet. Before, all pre-compiles were hard coded as part of the pallet. 1. Adding a pre-compile is as easy as implementing the new `Precompile` trait on any type. It can be added to the pallet by passing it into `Config::Precompiles`. This config know excepts a tuple of multiple types that each implement `Precompile`. 2. Each pre-compile has to implement Solidity ABI. Meaning its inputs and outputs are encoded according to Eth ABI. This makes writing a pre-compile much simpler since it doesn't have to implement its own decoding logic. More importantly: It makes it trivial to consume the API from a Solidity contract. 3. We constrain the address space of pre-compiles to a safe range so that they cannot accidentally match a wide range creating a collision with real contracts. 4. We check that pre-compile address ranges do not overlap at compile time. 5. Pre-compiles behave exactly as a normal contract. They exist as frames on the call stack and the environment they observe is their own (not the one of the calling contract). They can also be delegate called which changes the semantics in the same way as for normal contracts: They observe the environment of the calling contract. 6. They can also be called by the origin without any other contract in-between. Check the rustdocs of the `precompile` module on how to write a pre-compile. 1. A new module `precompiles` is added that contains the framework to write pre-compiles. It also contains the sub module `builtin` that contains hard coded pre-compiles which exist Ethereum. 2. The `pure_precompiles` module was deleted since all its pre-compiles were ported over to the new framework and are now housed in `builtin`. 4. The `CallSetup` type is moved outside of the `benchmarking` module because it is also needed for testing code now. It is intended to be used for implementors outside of the crate to test the pre-compiles (in addition to benchmarking them). - #8363 - #8364 - #8362 Fixes [#6716](#6716) --------- Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: PG Herveou <pgherveou@gmail.com> Co-authored-by: xermicus <cyrill@parity.io>

@kianenigma

This brings in `stable2506` Polkadot SDK, and integrates many new features. Integrated breaking changes to be verified by the original authors: - [x] ~paritytech/polkadot-sdk#8127 @kianenigma @Ank4n~ This will come in with AHM, and not before. - [x] paritytech/polkadot-sdk#7597 @gui1117 - [x] paritytech/polkadot-sdk#8254 @bkchr - [x] paritytech/polkadot-sdk#7592 @bkontur - [x] paritytech/polkadot-sdk#8382 @UtkarshBhardwaj007 - [x] paritytech/polkadot-sdk#8021 @serban300 - [x] paritytech/polkadot-sdk#8344 @serban300 - [x] paritytech/polkadot-sdk#8262 @athei - [x] paritytech/polkadot-sdk#8584 @athei - [x] paritytech/polkadot-sdk#8299 @skunert - [x] paritytech/polkadot-sdk#8652 @pgherveou - [x] paritytech/polkadot-sdk#8554 @pgherveou - [x] paritytech/polkadot-sdk#8281 @mrshiposha - [x] paritytech/polkadot-sdk#7730 @franciscoaguirre - [x] paritytech/polkadot-sdk#8599 @yrong @claravanstaden - [x] paritytech/polkadot-sdk#8531 @bkontur - [x] paritytech/polkadot-sdk#8409 @kianenigma - [x] paritytech/polkadot-sdk#9137 @franciscoaguirre - [x] paritytech/polkadot-sdk#7944 @bkontur - [x] paritytech/polkadot-sdk#8179 @bkontur - [x] paritytech/polkadot-sdk#8037 @yrong --------- Co-authored-by: GitHub Action <action@github.com> Co-authored-by: claravanstaden <claravanstaden64@gmail.com> Co-authored-by: Branislav Kontur <bkontur@gmail.com> Co-authored-by: Bastian Köcher <git@kchr.de> Co-authored-by: Alain Brenzikofer <alain@integritee.network> Co-authored-by: kianenigma <kian@parity.io> Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com> Co-authored-by: ron <yrong1997@gmail.com> Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com> Co-authored-by: Overkillus <maciej.zyszkiewicz@parity.io>

Add precompiles

1c48304

Update from github-actions[bot] running command 'fmt'

0e83666

Update from github-actions[bot] running command 'bench --runtime dev …

782894b

…--pallet pallet_revive'

athei added 2 commits April 16, 2025 23:02

Fix benchmarks

5fbd52b

Clippy

9514d33

Update from github-actions[bot] running command 'bench --runtime dev …

fa515d5

…--pallet pallet_revive'

athei added 7 commits April 16, 2025 23:48

Use new weight values

c36172d

Update penpal runtime

7eb7143

Fix docs

c551aa5

Fix tests

f121424

Don't export macros from crate and add some docs

0811818

Add run::precompile

4d5e4a5

Add more docs

85e1180

pgherveou reviewed Apr 22, 2025

View reviewed changes

athei and others added 4 commits April 23, 2025 16:27

Zero out output buffer len on error

bec4ea9

Add more tests for pre-compiles

9687819

Merge branch 'master' into at/precompiles

57856c4

Apply suggestions from code review

f0c95b7

Co-authored-by: PG Herveou <pgherveou@gmail.com>

pgherveou reviewed Apr 24, 2025

View reviewed changes

athei and others added 3 commits April 24, 2025 15:39

Apply suggestions from code review

3ecb05b

Co-authored-by: PG Herveou <pgherveou@gmail.com>

Merge branch 'master' into at/precompiles

796c388

Fix some docs

7cdfacd

xermicus approved these changes Apr 25, 2025

View reviewed changes

substrate/frame/revive/src/exec.rs Show resolved Hide resolved

substrate/frame/revive/src/precompiles.rs Outdated Show resolved Hide resolved

substrate/frame/revive/src/precompiles/tests.rs Outdated Show resolved Hide resolved

athei and others added 2 commits April 25, 2025 18:34

Apply suggestions from code review

379335f

Co-authored-by: xermicus <cyrill@parity.io>

Merge branch 'master' into at/precompiles

62db3dc

athei added 2 commits April 28, 2025 17:46

Only charge execution from the nested meter

23c3ed2

Merge branch 'master' into at/precompiles

47149b2

Disable broken test

863c4da

athei enabled auto-merge April 28, 2025 17:15

athei added this pull request to the merge queue Apr 28, 2025

Merged via the queue into master with commit 398bbfc Apr 28, 2025
239 of 249 checks passed

athei deleted the at/precompiles branch April 28, 2025 18:01

acatangiu mentioned this pull request Jul 16, 2025

Upgrade to Polkadot-SDK stable2506 polkadot-fellows/runtimes#817

Merged

21 tasks

github-actions bot mentioned this pull request Oct 24, 2025

Update polkadot-sdk from stable2503 to stable2506 moondance-labs/tanssi#1340

Closed

pgherveou added this to [preview] release tracker Mar 20, 2026

github-project-automation bot moved this to Todo in [preview] release tracker Mar 20, 2026

github-project-automation bot moved this from Todo to Done in [preview] release tracker Mar 20, 2026

pgherveou removed this from [preview] release tracker Mar 20, 2026

pgherveou added this to [preview] release tracker Mar 23, 2026

github-project-automation bot moved this to Todo in [preview] release tracker Mar 23, 2026

github-project-automation bot moved this from Todo to Done in [preview] release tracker Mar 23, 2026

Conversation

athei commented Apr 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Design

Changes

Follow Ups

Uh oh!

athei commented Apr 16, 2025

Uh oh!

athei commented Apr 16, 2025

Uh oh!

github-actions bot commented Apr 16, 2025

Uh oh!

github-actions bot commented Apr 16, 2025

Uh oh!

athei commented Apr 16, 2025

Uh oh!

github-actions bot commented Apr 16, 2025

Uh oh!

github-actions bot commented Apr 16, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pgherveou commented Apr 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

xermicus left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

athei commented Apr 25, 2025

Uh oh!

xermicus commented Apr 25, 2025

Uh oh!

athei commented Apr 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

xermicus commented Apr 26, 2025

Uh oh!

athei commented Apr 26, 2025

Uh oh!

athei commented Apr 28, 2025

Uh oh!

paritytech-workflow-stopper bot commented Apr 28, 2025

Uh oh!

Uh oh!

xermicus commented May 5, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

athei commented Apr 16, 2025 •

edited

Loading

pgherveou commented Apr 24, 2025 •

edited

Loading

athei commented Apr 26, 2025 •

edited

Loading