Skip to content

Comments

simulators/ethereum/pyspec: Add simulator for python spec tests (ethereum/exec-spec-tests)#732

Merged
marioevz merged 20 commits intoethereum:masterfrom
spencer-tb:simulators/ethereum/pyspec
Mar 15, 2023
Merged

simulators/ethereum/pyspec: Add simulator for python spec tests (ethereum/exec-spec-tests)#732
marioevz merged 20 commits intoethereum:masterfrom
spencer-tb:simulators/ethereum/pyspec

Conversation

@spencer-tb
Copy link
Collaborator

@spencer-tb spencer-tb commented Feb 23, 2023

Adds a new simulator for the tests in the execution-spec-tests repo: https://github.com/ethereum/execution-spec-tests

For each test fixture:

  • Loads genesis fields into client,
  • Blocks are fed into client one by one using EngineAPI,
  • Storage, Nonce, Balance is extracted from last valid block using EngineAPI, and compared against expected fixture values.

Only for fixtures with forks >= Merge.

@spencer-tb spencer-tb marked this pull request as ready for review February 23, 2023 21:29
marioevz
marioevz requested changes Feb 23, 2023
View reviewed changes
Copy link
Member

@marioevz marioevz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks very promising!

A few comments.

spencer-tb and others added 3 commits February 24, 2023 14:08
@spencer-tb
simulators/ethereum/pyspec: Update types file (post-review change).
2114867
@spencer-tb
simulators/ethereum/pyspec: Add INVALID/VALID status checks, Dockefil…
e42d8c0 
…e update, etc (post-review change).
@fjl
simulators/ethereum/pyspec: remove simulator code description
7db8027 
There is no need to document in such detail how the simulator works,
since it's basically the same for all other simulators in this repo.
@fjl
Copy link
Collaborator

fjl commented Feb 27, 2023

I have removed the very detailed documentation in README.md. There is certainly no need to duplicate parts of the code into the README.

fjl
fjl reviewed Feb 28, 2023
View reviewed changes
fjl
fjl reviewed Feb 28, 2023
View reviewed changes
@spencer-tb spencer-tb mentioned this pull request Feb 28, 2023
Merged
fjl
fjl reviewed Feb 28, 2023
View reviewed changes
@spencer-tb spencer-tb requested a review from marioevz March 8, 2023 18:27
marioevz
marioevz reviewed Mar 15, 2023
View reviewed changes
Copy link
Member

@marioevz marioevz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some more comments.

simulators/ethereum/pyspec/types.go Outdated
// ------------------------------------------------------------------------------ //
func extractGenesis(fixture fixtureJSON) *core.Genesis {
genesis := &core.Genesis{
Timestamp: fixture.Genesis.Timestamp.Uint64(),
Copy link
Member

@marioevz marioevz Mar 1, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we sort these fields in the same order as they are defined in the types.Header definition ?
The types.Header order follows the actual order of the fields when they are RLP encoded.

simulators/ethereum/pyspec/types.go Outdated
//go:generate go run github.com/fjl/gencodec -type transactions -field-override transactionsUnmarshaling -out gen_txs.go
type transactions struct {
Nonce uint64 `json:"nonce"`
To common.Address `json:"to.omitempty"`
Copy link
Member

@marioevz marioevz Mar 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
To common.Address `json:"to.omitempty"`
To common.Address `json:"to,omitempty"`

simulators/ethereum/pyspec/types.go Outdated
}

//go:generate go run github.com/fjl/gencodec -type transactions -field-override transactionsUnmarshaling -out gen_txs.go
type transactions struct {
Copy link
Member

@marioevz marioevz Mar 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Type name should be transaction, because normally the plural is reserved for a type that overrides an array.

simulators/ethereum/pyspec/types.go
GasLimit uint64 `json:"gasLimit"`
GasUsed uint64 `json:"gasUsed"`
SecretKey common.Hash `json:"secretKey"`
}
Copy link
Member

@marioevz marioevz Mar 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some missing fields that should be included from here:
https://github.com/ethereum/execution-spec-tests/blob/2dcfb503f4c50d368958c34a99b39e1aa3f62238/src/ethereum_test_tools/common/types.py#L912-L940

simulators/ethereum/pyspec/types.go Outdated
TransactionsTrie common.Hash `json:"transactionsRoot"`
TransactionsTrieAlt common.Hash `json:"transactionsTrie"` // name in fixtures
ReceiptTrie common.Hash `json:"receiptsRoot"`
ReceiptTrieAlt common.Hash `json:"receiptTrie"` // name in fixturse
Copy link
Member

@marioevz marioevz Mar 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ReceiptTrieAlt common.Hash `json:"receiptTrie"` // name in fixturse
ReceiptTrieAlt common.Hash `json:"receiptTrie"` // name in fixtures

@marioevz marioevz merged commit 7d35d47 into ethereum:master Mar 15, 2023
mattsse pushed a commit to mattsse/hive that referenced this pull request Mar 24, 2023
@spencer-tb @fjl
@marioevz @mattsse
simulators/ethereum/pyspec: Add simulator for python spec tests (ethe…
b882f0a 
…reum/exec-spec-tests) (ethereum#732)

* simulators/ethereum/pyspec: running with EngineAPI.

* simulators/ethereum/pyspec: Update types file (post-review change).

* simulators/ethereum/pyspec: Add INVALID/VALID status checks, Dockefile update, etc (post-review change).

* simulators/ethereum/pyspec: remove simulator code description

There is no need to document in such detail how the simulator works,
since it's basically the same for all other simulators in this repo.

* simulators/ethereum/pyspec: Correctly use gencodec for types (post-review change).

* simulators/ethereum/engine: add method to get storage from multiple accounts

* simulators/ethereum: workspace

* simulators/ethereum: workspace go.work.sum

* simulators/ethereum/pyspec: update custom hive_rpc function to StorageAtKeys (post-review change).

* simulators/ethereum/pyspec: address comments, use StorageAtKeys for batch calls (post-review change).

* simulators/ethereum/pyspec: add option to exclude or run specific test fixtures.

* simulators/ethereum/pyspec: remove go work files in ethereum/*.

* simulators/ethereum/pyspec: re-add go work files in ethereum/* with all simulators.

* simulators/ethereum/pyspec: get --sim-limit working within simulator.

* simulators/ethereum/pyspec: testname update, working on geth, erigon, besu.

* small tweak to fixture names, failed tests are output at the end of simulation.

* Address comments for types.go, add all fields for transactions.

* .

* Fix for transaction marshalling.

---------

Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: marioevz <marioevz@gmail.com>
racytech pushed a commit to racytech/hive that referenced this pull request Apr 4, 2025
@barnabasbusa
fix: add debug ns to nethermind (ethereum#732)
372bb52
racytech pushed a commit to racytech/hive that referenced this pull request Apr 4, 2025
@EthPandaOpsBot
chore(main): release 4.2.0 (ethereum#709)
2e9e5a4 
🤖 I have created a release *beep* *boop*
---


##
[4.2.0](ethpandaops/ethereum-package@4.1.0...4.2.0)
(2024-08-19)


### Features

* add customizable configuraiton for prometheus retention
([ethereum#745](ethpandaops/ethereum-package#745))
([6c02dfe](ethpandaops/ethereum-package@6c02dfe))
* add genesis_gaslimit param configuration
([ethereum#726](ethpandaops/ethereum-package#726))
([a4ba9a6](ethpandaops/ethereum-package@a4ba9a6))
* Add Lighthouse PeerDAS Dashboard
([ethereum#736](ethpandaops/ethereum-package#736))
([ffbfde2](ethpandaops/ethereum-package@ffbfde2))
* add peerdas-electra-support
([ethereum#740](ethpandaops/ethereum-package#740))
([663e7e6](ethpandaops/ethereum-package@663e7e6))
* add profiling for prysm
([ethereum#722](ethpandaops/ethereum-package#722))
([7dc6660](ethpandaops/ethereum-package@7dc6660))
* add sanity check
([ethereum#710](ethpandaops/ethereum-package#710))
([b824cac](ethpandaops/ethereum-package@b824cac))
* enable dora pprof
([ethereum#743](ethpandaops/ethereum-package#743))
([2b7be9a](ethpandaops/ethereum-package@2b7be9a))
* generate keys if not default key is used
([ethereum#707](ethpandaops/ethereum-package#707))
([2d1cab5](ethpandaops/ethereum-package@2d1cab5))
* lodestar persists invalid ssz objects by default
([ethereum#730](ethpandaops/ethereum-package#730))
([5a45991](ethpandaops/ethereum-package@5a45991))
* update egg config parameters
([ethereum#737](ethpandaops/ethereum-package#737))
([78c2bc7](ethpandaops/ethereum-package@78c2bc7))


### Bug Fixes

* add debug ns to nethermind
([ethereum#732](ethpandaops/ethereum-package#732))
([372bb52](ethpandaops/ethereum-package@372bb52))
* allow vc properties in participants_matrix.cl
([ethereum#715](ethpandaops/ethereum-package#715))
([c8b9b19](ethpandaops/ethereum-package@c8b9b19))
* besu devnet schedule
([ethereum#734](ethpandaops/ethereum-package#734))
([28b67cd](ethpandaops/ethereum-package@28b67cd))
* blobscan redis dependency
([ethereum#712](ethpandaops/ethereum-package#712))
([0ed1c9c](ethpandaops/ethereum-package@0ed1c9c))
* change churn limit default for pectra tests
([ethereum#747](ethpandaops/ethereum-package#747))
([8109054](ethpandaops/ethereum-package@8109054))
* correctly apply extra params if builder is enabled
([ethereum#725](ethpandaops/ethereum-package#725))
([a94caf0](ethpandaops/ethereum-package@a94caf0))
* disable all assertoor tests by default
([ethereum#738](ethpandaops/ethereum-package#738))
([2961f96](ethpandaops/ethereum-package@2961f96))
* lodestar vc faster startup
([ethereum#721](ethpandaops/ethereum-package#721))
([225e3d8](ethpandaops/ethereum-package@225e3d8))
* readme eof
([ethereum#739](ethpandaops/ethereum-package#739))
([7f94f6e](ethpandaops/ethereum-package@7f94f6e))
* remove custom peerdas images for dora & assertoor
([ethereum#741](ethpandaops/ethereum-package#741))
([a19398d](ethpandaops/ethereum-package@a19398d))
* remove subscribe all subnet nimbus
([ethereum#719](ethpandaops/ethereum-package#719))
([ef92f8f](ethpandaops/ethereum-package@ef92f8f))
* remove subscribe-all-subnets from default prysm config
([ethereum#717](ethpandaops/ethereum-package#717))
([6348c0b](ethpandaops/ethereum-package@6348c0b))
* remove subscribe-all-subnets lighthouse/lodestar/teku
([ethereum#720](ethpandaops/ethereum-package#720))
([cdb20e1](ethpandaops/ethereum-package@cdb20e1))
* update snapshots URL
([ethereum#731](ethpandaops/ethereum-package#731))
([f9269ad](ethpandaops/ethereum-package@f9269ad))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@fjl fjl fjl left review comments

@marioevz marioevz marioevz approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@spencer-tb @fjl @marioevz