-
Notifications
You must be signed in to change notification settings - Fork 953
Reduce eth2 dependency space
#8524
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
We could also consider feature gating all SSZ related crates/dependencies if we can. The http client would then basically be JSON-only which is probably suitable for many use cases, and might even be required for WASM support |
michaelsproul
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great! Definitely a step in the right direction!
| default = ["lighthouse"] | ||
| lighthouse = [] | ||
| default = [] | ||
| lighthouse = ["proto_array", "eth2_keystore", "eip_3076", "zeroize"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Another thing we could do is split the "standard" VC endpoints and Lighthouse VC endpoints. At the moment both are under the same feature as lighthouse. Only the standard VC endpoints require eip_3076, for example.
| } | ||
|
|
||
| #[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize, Encode, Decode, TestRandom)] | ||
| #[cfg_attr(test, derive(TestRandom))] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just noting that this attribute means we will not be able to use TestRandom on BlobsBundle in any crates that depend on eth2, because dependencies do not get compiled with cfg(test):
This is fine at the moment, but might need to be adjusted in future. And in general I think we should be wary of putting any actual code behind cfg(test). A separate cfg(feature = "testing") is the alternative.
| //! This module exposes a superset of the `types` crate. It adds additional types that are only | ||
| //! required for the HTTP API. | ||
| pub use types::*; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like that the git blame for this line will now be your username 😉
Merge Queue Status✅ The pull request has been merged This pull request spent 39 minutes 47 seconds in the queue, including 37 minutes 54 seconds running CI. Required conditions to merge
|
* Update local testnet scripts for the fulu fork (sigp#8489) * Remove `fulu-devnet-3` testing on CI * Delete `scripts/local_testnet/network_params_das.yaml` and consolidate it into the main `network_params.yaml` file we use on CI * Delete enclave before building image, so it doesn't cause slow image building. Co-Authored-By: Jimmy Chen <[email protected]> * Fix data columns sorting when reconstructing blobs (sigp#8510) Closes sigp#8509 Co-Authored-By: Antoine James <[email protected]> * Instrument attestation signing. (sigp#8508) We noticed attestation signing taking 2+ seconds on some of our hoodi nodes and the current traces doesn't provide enough details. This PR adds a few more spans to the `attestation_duty_cycle` code path in the VC. Before: <img width="842" height="154" alt="image" src="https://github.com/user-attachments/assets/cfc5c8c0-e6f2-4f56-a8e4-65001af4a005" /> After: <img width="496" height="217" alt="image" src="https://github.com/user-attachments/assets/c91cfa88-af1b-456e-8c64-625809eb6881" /> Co-Authored-By: Jimmy Chen <[email protected]> * Always use committee index 0 when getting attestation data (sigp#8171) * sigp#8046 Split the function `publish_attestations_and_aggregates` into `publish_attestations` and `handle_aggregates`, so that for attestations, only 1 task is spawned. Co-Authored-By: Tan Chee Keong <[email protected]> Co-Authored-By: chonghe <[email protected]> Co-Authored-By: Michael Sproul <[email protected]> Co-Authored-By: Michael Sproul <[email protected]> * Move deposit contract artifacts to /target (sigp#8518) Alternative to: - sigp#8488 Refactors deposit_contract crate to comply with Rust build conventions by placing generated artifacts in the build output directory. Co-Authored-By: Michael Sproul <[email protected]> * Move beacon state endpoints to a separate module. (sigp#8529) Part of the http api refactor to move endpoint handlers to separate modules. This should improve code maintainability, incremental compilation time and rust analyzer performance. Co-Authored-By: Jimmy Chen <[email protected]> * Refactor `consensus/types` (sigp#7827) Organize and categorize `consensus/types` into modules based on their relation to key consensus structures/concepts. This is a precursor to a sensible public interface. While this refactor is very opinionated, I am open to suggestions on module names, or type groupings if my current ones are inappropriate. Co-Authored-By: Mac L <[email protected]> * Move validator http endpoints to a separate module (sigp#8536) Continuation of: * sigp#8529 Moving `/validator` endpoints out of `http_api` to a separation module. This should improve code maintainability, incremental compilation time and rust analyzer performance. This is a tedious but straight forward change, so we're going with a pair & insta-merge approach to avoid painful & slow async review. @michaelsproul and I paired on the first commit - I believe we are almost done, will pair with @pawanjay176 tomorrow to wrap it up and merge tomorrow. (cc @macladson ) Co-Authored-By: Jimmy Chen <[email protected]> * Move beacon pool http api to its own separate module (sigp#8543) Continuation of: * sigp#8536 Moving `/beacon/pool` endpoints out of `http_api` to a separation module. This should improve code maintainability, incremental compilation time and rust analyzer performance. This is a tedious but straight forward change, so we're going with a pair & insta-merge approach to avoid painful & slow async review Co-Authored-By: Jimmy Chen <[email protected]> * Reduce `eth2` dependency space (sigp#8524) Remove certain dependencies from `eth2`, and feature-gate others which are only used by certain endpoints. | Removed | Optional | Dev only | | -------- | -------- | -------- | | `either` `enr` `libp2p-identity` `multiaddr` | `protoarray` `eth2_keystore` `eip_3076` `zeroize` `reqwest-eventsource` `futures` `futures-util` | `rand` `test_random_derive` | This is done by adding an `events` feature which enables the events endpoint and its associated dependencies. The `lighthouse` feature also enables its associated dependencies making them optional. The networking-adjacent dependencies were removed by just having certain fields use a `String` instead of an explicit network type. This means the user should handle conversion at the call site instead. This is a bit spicy, but I believe `PeerId`, `Enr` and `Multiaddr` are easily converted to and from `String`s so I think it's fine and reduces our dependency space by a lot. The alternative is to feature gate these types behind a `network` feature instead. Co-Authored-By: Mac L <[email protected]> * Clarify `alloy` dependencies (sigp#8550) Previously, we had a pinned version of `alloy` to fix some crate compatibility issues we encountered during the migration away from `ethers`. Now that the migration is complete we should remove the pin. This also updates alloy crates to their latest versions. Co-Authored-By: Mac L <[email protected]> * Remove `consensus/types` re-exports (sigp#8540) There are certain crates which we re-export within `types` which creates a fragmented DevEx, where there are various ways to import the same crates. ```rust // consensus/types/src/lib.rs pub use bls::{ AggregatePublicKey, AggregateSignature, Error as BlsError, Keypair, PUBLIC_KEY_BYTES_LEN, PublicKey, PublicKeyBytes, SIGNATURE_BYTES_LEN, SecretKey, Signature, SignatureBytes, get_withdrawal_credentials, }; pub use context_deserialize::{ContextDeserialize, context_deserialize}; pub use fixed_bytes::FixedBytesExtended; pub use milhouse::{self, List, Vector}; pub use ssz_types::{BitList, BitVector, FixedVector, VariableList, typenum, typenum::Unsigned}; pub use superstruct::superstruct; ``` This PR removes these re-exports and makes it explicit that these types are imported from a non-`consensus/types` crate. Co-Authored-By: Mac L <[email protected]> * Fix testnet script (sigp#8557) Fix an issue where a kurtosis testnet script was failing because no supernodes were provided ``` There was an error interpreting Starlark code Evaluation error: fail: Fulu fork is enabled (epoch: 0) but no supernodes are configured, no nodes have 128 or more validators, and perfect_peerdas_enabled is not enabled. Either configure a supernode, ensure at least one node has 128+ validators, or enable perfect_peerdas_enabled in network_params with 16 participants. at [github.com/ethpandaops/ethereum-package/main.star:83:57]: run at [github.com/ethpandaops/ethereum-package/src/package_io/input_parser.star:377:17]: input_parser at [0:0]: fail ``` Co-Authored-By: Eitan Seri-Levi <[email protected]> Co-Authored-By: Pawan Dhananjay <[email protected]> * Do not request attestation data when attestation duty is empty (sigp#8559) Co-Authored-By: Tan Chee Keong <[email protected]> * Rust 1.92 lints (sigp#8567) Co-Authored-By: Eitan Seri-Levi <[email protected]> --------- Co-authored-by: Jimmy Chen <[email protected]> Co-authored-by: Jimmy Chen <[email protected]> Co-authored-by: 0xMushow <[email protected]> Co-authored-by: Antoine James <[email protected]> Co-authored-by: chonghe <[email protected]> Co-authored-by: Michael Sproul <[email protected]> Co-authored-by: Michael Sproul <[email protected]> Co-authored-by: Mac L <[email protected]> Co-authored-by: Eitan Seri-Levi <[email protected]> Co-authored-by: Pawan Dhananjay <[email protected]> Co-authored-by: Tan Chee Keong <[email protected]>
Proposed Changes
Remove certain dependencies from
eth2, and feature-gate others which are only used by certain endpoints.eitherenrlibp2p-identitymultiaddrprotoarrayeth2_keystoreeip_3076zeroizereqwest-eventsourcefuturesfutures-utilrandtest_random_deriveThis is done by adding an
eventsfeature which enables the events endpoint and its associated dependencies.The
lighthousefeature also enables its associated dependencies making them optional.The networking-adjacent dependencies were removed by just having certain fields use a
Stringinstead of an explicit network type. This means the user should handle conversion at the call site instead. This is a bit spicy, but I believePeerId,EnrandMultiaddrare easily converted to and fromStrings so I think it's fine and reduces our dependency space by a lot. The alternative is to feature gate these types behind anetworkfeature instead.Additional Info
Here are some
cargo treestats before and after these changes: