Skip to content

Comments

simulators/eth2/common: create client interfaces, update mock builder#738

Merged
marioevz merged 11 commits intoethereum:masterfrom
marioevz:refactor-clients
Apr 13, 2023
Merged

simulators/eth2/common: create client interfaces, update mock builder#738
marioevz merged 11 commits intoethereum:masterfrom
marioevz:refactor-clients

Conversation

@marioevz
Copy link
Member

@marioevz marioevz commented Mar 3, 2023 •
edited
Loading

This PR introduces the interfaces Client and ManagedClient which are now used by the ExecutionClient, BeaconClient and ValidatorClient structures to communicate with the clients.

Previously, the structures directly used hivesim.Client, which required a client launched and managed by hive.

hivesim.Client is now abstracted through ManagedClient to allow instantiation of ExecutionClient, BeaconClient or ValidatorClient and use an external client.

With this change, modules like the mock builder can now be imported by other non-hive modules, and even compiled separately as standalone commands.

Changes Included

Testnet Clients

  • Client/ManagedClient interfaces
  • Execution and Beacon clients now have separate configuration structures, ExecutionClientConfig and BeaconClientConfig respectively
  • Beacon client structure can now automatically fetch the beacon spec from the client itself
  • Beacon client subnets are now possible: If a beacon client is started with a specific subnet, their ENR will only be passed to the clients of the same subnet (useful to test forks and re-orgs)
  • Testnet creation now uses configuration structs

Withdrawals Test Suite

  • Tests now use the mock builder functionality to invalidate payloads

Mock builder

  • Support to invalidate payloads, used in testnets to verify the circuit breaker behavior
  • Support to create invalid outdated (from a previous fork) payloads.
  • Mock builder behavior can now be altered while running by using the REST API (see builder/mock/README.md)

@holgerd77
Copy link
Contributor

holgerd77 commented Mar 6, 2023

Hi @marioevz, thanks for all these extensive restructuring/refactoring works and efforts, really appreciated by "the users"!! 🤗 🤩 ❤️

@marioevz marioevz requested a review from fjl March 13, 2023 14:35
@marioevz
Copy link
Member Author

marioevz commented Mar 14, 2023

We could first merge #741 and then I could update the ManagedClient interface in this PR to include pause/unpause feature, which would allow to construct beacon chain tests which pause the beacon and validator clients. But let me know what you think @fjl

@marioevz
Copy link
Member Author

marioevz commented Apr 13, 2023

I've been running these changes locally for quite some time now, and they are required to introduce more tests into the branch, so I will be merging now.

@marioevz marioevz merged commit 28a27c9 into ethereum:master Apr 13, 2023
mattsse pushed a commit to paradigmxyz/hive that referenced this pull request May 12, 2023
@shekhirin @marioevz
@gfukushima @yperbasis @s1na @spencer-tb @holiman
Merge ethereum/hive/master (#26)
bbd52d3 
* simulators/ethereum/engine: send `eth_` requests to port 8545 (ethereum#752)

simulators/ethereum/engine: send `eth_` requests to 8545

* clients/ethereumjs: Update branch (ethereum#753)

* clients/ethereumjs: Change git branch

* clients/ethereumjs: Fix bash script issues

* simulators/ethereum/engine: node: unsubscribe from events

* simulators/ethereum/engine: clmock: add extra logs

* simulators/ethereum/engine: correctly stop secondary clients

* simulators/eth2/common: create client interfaces, update mock builder (ethereum#738)

* simulators/eth2/common: refactor clients

* simulators/eth2/engine: fixes due to refactor

* simulators/eth2/common: builder: invalidate payloads

* simulators/eth2/common: builder readme

* simulators/eth2/common: builder: add option to build a payload on an outdated version

* simulators/eth2/withdrawals: add invalid built payload version test

* simulators/eth2/common: builder: remove check to build invalid payload

* simulators/eth2/withdrawals: Fix check

* simulators/eth2/withdrawals: test case validation

* simulators/eth2/common: builder: fix invalidation

* simulators/eth2/withdrawals: fix verifications again

* simulators/ethereum/pyspec: Comment Dockerfile to run local fixtures (ethereum#755)

simulators/ethereum/pyspec: Add Dockerfile comment to run locally generated fixtures

* clients/besu: add --data-storage-format=BONSAI  (ethereum#758)

* simulators/ethereum/engine: Unique Payload ID Tests (ethereum#757)

* simulators/ethereum/engine: expose generic fcu, np

* simulators/ethereum/engine: Unique payload id test

* simulators/ethereum/engine: Add payload id test

* simulators/ethereum/engine: Parallel getPayloadBodies Test (ethereum#749)

* simulators/ethereum/engine: parallel getpayloadbodies

* simulators/ethereum/withdrawals: readme

* clients/erigon: --externalcl flag is not supported anymore (ethereum#760)

* graphql: consistent rule for Long outputs and args (ethereum#746)

* simulators/eth2: Verify block signatures on blocks sent to builder (ethereum#756)

* simulators/eth2/common/builder: interfaces changes

* simulators/eth2/common/builder: verify signatures

* simulators/eth2/withdrawals: Verify block signatures

* simulators/eth2/withdrawals: Update README

* simulators/eth2/withdrawals: minor nit comment

* clients/erigon: explicit --db.size.limit value (ethereum#766)

* clients/nethermind: Cancun support (ethereum#762)

* clients/nethermind: Cancun Support

* clients/nethermind: Dockerfile args

* clients/besu: Cancun Support (ethereum#769)

* clients/ethereumjs: Cancun Support (ethereum#768)

clients/ethereumjs: Cancun Support.

* clients/go-ethereum: remove deprecated PoW (ethereum#773)

Remove ethash-related flags and utils from go-ethereum, since they were
deprecated upstream.

---------

Co-authored-by: Martin Holst Swende <martin@swende.se>

* clients/nethermind: Add excessDataGas to genesis mapper. (ethereum#771)

* sim/pyspec: bubble error (ethereum#777)

* simulators/devp2p: fix issue causing ethereumjs to crash (ethereum#778)

sim/devp2p: fix issue causing ethereumjs to crash

It seems ethereumjs crashes if the chainid is not present in the genesis.

---------

Co-authored-by: Mario Vega <marioevz@gmail.com>
Co-authored-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
Co-authored-by: Andrew Ashikhmin <34320705+yperbasis@users.noreply.github.com>
Co-authored-by: Sina Mahmoodi <1591639+s1na@users.noreply.github.com>
Co-authored-by: Spencer Taylor-Brown <spencer@spencertaylorbrown.uk>
Co-authored-by: Martin Holst Swende <martin@swende.se>
This was referenced Jul 17, 2023
Merged
Merged
racytech pushed a commit to racytech/hive that referenced this pull request Apr 4, 2025
@pk910
fix: disable all assertoor tests by default (ethereum#738)
2961f96 
disable all assertoor tests by default to prevent accidentally running
tests twice
racytech pushed a commit to racytech/hive that referenced this pull request Apr 4, 2025
@EthPandaOpsBot
chore(main): release 4.2.0 (ethereum#709)
2e9e5a4 
🤖 I have created a release *beep* *boop*
---


##
[4.2.0](ethpandaops/ethereum-package@4.1.0...4.2.0)
(2024-08-19)


### Features

* add customizable configuraiton for prometheus retention
([ethereum#745](ethpandaops/ethereum-package#745))
([6c02dfe](ethpandaops/ethereum-package@6c02dfe))
* add genesis_gaslimit param configuration
([ethereum#726](ethpandaops/ethereum-package#726))
([a4ba9a6](ethpandaops/ethereum-package@a4ba9a6))
* Add Lighthouse PeerDAS Dashboard
([ethereum#736](ethpandaops/ethereum-package#736))
([ffbfde2](ethpandaops/ethereum-package@ffbfde2))
* add peerdas-electra-support
([ethereum#740](ethpandaops/ethereum-package#740))
([663e7e6](ethpandaops/ethereum-package@663e7e6))
* add profiling for prysm
([ethereum#722](ethpandaops/ethereum-package#722))
([7dc6660](ethpandaops/ethereum-package@7dc6660))
* add sanity check
([ethereum#710](ethpandaops/ethereum-package#710))
([b824cac](ethpandaops/ethereum-package@b824cac))
* enable dora pprof
([ethereum#743](ethpandaops/ethereum-package#743))
([2b7be9a](ethpandaops/ethereum-package@2b7be9a))
* generate keys if not default key is used
([ethereum#707](ethpandaops/ethereum-package#707))
([2d1cab5](ethpandaops/ethereum-package@2d1cab5))
* lodestar persists invalid ssz objects by default
([ethereum#730](ethpandaops/ethereum-package#730))
([5a45991](ethpandaops/ethereum-package@5a45991))
* update egg config parameters
([ethereum#737](ethpandaops/ethereum-package#737))
([78c2bc7](ethpandaops/ethereum-package@78c2bc7))


### Bug Fixes

* add debug ns to nethermind
([ethereum#732](ethpandaops/ethereum-package#732))
([372bb52](ethpandaops/ethereum-package@372bb52))
* allow vc properties in participants_matrix.cl
([ethereum#715](ethpandaops/ethereum-package#715))
([c8b9b19](ethpandaops/ethereum-package@c8b9b19))
* besu devnet schedule
([ethereum#734](ethpandaops/ethereum-package#734))
([28b67cd](ethpandaops/ethereum-package@28b67cd))
* blobscan redis dependency
([ethereum#712](ethpandaops/ethereum-package#712))
([0ed1c9c](ethpandaops/ethereum-package@0ed1c9c))
* change churn limit default for pectra tests
([ethereum#747](ethpandaops/ethereum-package#747))
([8109054](ethpandaops/ethereum-package@8109054))
* correctly apply extra params if builder is enabled
([ethereum#725](ethpandaops/ethereum-package#725))
([a94caf0](ethpandaops/ethereum-package@a94caf0))
* disable all assertoor tests by default
([ethereum#738](ethpandaops/ethereum-package#738))
([2961f96](ethpandaops/ethereum-package@2961f96))
* lodestar vc faster startup
([ethereum#721](ethpandaops/ethereum-package#721))
([225e3d8](ethpandaops/ethereum-package@225e3d8))
* readme eof
([ethereum#739](ethpandaops/ethereum-package#739))
([7f94f6e](ethpandaops/ethereum-package@7f94f6e))
* remove custom peerdas images for dora & assertoor
([ethereum#741](ethpandaops/ethereum-package#741))
([a19398d](ethpandaops/ethereum-package@a19398d))
* remove subscribe all subnet nimbus
([ethereum#719](ethpandaops/ethereum-package#719))
([ef92f8f](ethpandaops/ethereum-package@ef92f8f))
* remove subscribe-all-subnets from default prysm config
([ethereum#717](ethpandaops/ethereum-package#717))
([6348c0b](ethpandaops/ethereum-package@6348c0b))
* remove subscribe-all-subnets lighthouse/lodestar/teku
([ethereum#720](ethpandaops/ethereum-package#720))
([cdb20e1](ethpandaops/ethereum-package@cdb20e1))
* update snapshots URL
([ethereum#731](ethpandaops/ethereum-package#731))
([f9269ad](ethpandaops/ethereum-package@f9269ad))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@fjl fjl Awaiting requested review from fjl

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@marioevz @holgerd77