Created Fulu skeleton setup#9325
Conversation
| # Fulu | ||
| FULU_FORK_VERSION: 0x07017000 | ||
| FULU_FORK_EPOCH: 18446744073709551615 | ||
|
|
There was a problem hiding this comment.
we should actually be able to put all the fulu fields in and be standard to the testconfig again iirc...
| case FULU -> | ||
| builder -> | ||
| builder | ||
| .altairBuilder(a -> a.altairForkEpoch(UInt64.ZERO)) | ||
| .bellatrixBuilder(b -> b.bellatrixForkEpoch(UInt64.ZERO)) | ||
| .capellaBuilder(c -> c.capellaForkEpoch(UInt64.ZERO)) | ||
| .denebBuilder(d -> d.denebForkEpoch(UInt64.ZERO)) | ||
| .electraBuilder(e -> e.electraForkEpoch(UInt64.ZERO)) | ||
| .fuluBuilder(f -> f.fuluForkEpoch(UInt64.ZERO)); |
There was a problem hiding this comment.
i do wonder if we could just refactor this all to be less ugly... that whole pattern of for each new fork we add a line we could clean up...
something like
if (specMilestone>= ALTAIR) {
builder -> builder.altairBuilder(a -> a.altairForkEpoch(UInt64.ZERO));
}
if (specMilestone>= BELLATRIX) {
builder -> builder.bellatrixBuilder(a -> a.bellatrixForkEpoch(UInt64.ZERO));
}
There was a problem hiding this comment.
I have made a change similar to what you suggested. My only concern is that we might end up missing this when we add a new fork. Using the switch we get a compiler time warning when we are missing a fork. Wdyt?
There was a problem hiding this comment.
put a 'if last enum is greater than fulu' test... :)
|
|
||
| @Override | ||
| public int getMaxBlobsPerBlockFulu() { | ||
| return maxBlobsPerBlockFulu; |
There was a problem hiding this comment.
We have this in SpecConfigElectraImpl. My assumptiions was that we should continue in the same style:
@Override
public int getMaxBlobsPerBlock() {
return maxBlobsPerBlockElectra;
}| # Misc | ||
| # --------------------------------------------------------------- | ||
| # `uint64(12)` | ||
| MAX_BLOBS_PER_BLOCK_FULU: 12 |
There was a problem hiding this comment.
It's a part of config, not preset, see https://github.com/ethereum/consensus-specs/blob/dev/configs/mainnet.yaml#L180
The main differrence is that config values can be overriden by custom config for example in Kurtosis, presets cannot
| KZG_COMMITMENTS_INCLUSION_PROOF_DEPTH: 4 | ||
|
|
||
| # Not yet in official spec | ||
| MIN_EPOCHS_FOR_DATA_COLUMN_SIDECARS_REQUESTS: 4096 No newline at end of file |
There was a problem hiding this comment.
It's a part of config also, see: https://github.com/ethereum/consensus-specs/blob/dev/configs/mainnet.yaml#L181
| # Misc | ||
| # --------------------------------------------------------------- | ||
| # `uint64(12)` | ||
| MAX_BLOBS_PER_BLOCK_FULU: 12 |
There was a problem hiding this comment.
Same here and in swift. Shouldn't be a part of preset
| tekuNodeConfigBuilder.withElectraEpoch(UInt64.ZERO); | ||
| break; | ||
| case FULU: | ||
| tekuNodeConfigBuilder.withFuluEpoch(UInt64.ZERO); |
There was a problem hiding this comment.
Looking at previous milestone configurations, shouldn't it be
tekuNodeConfigBuilder.withCapellaEpoch(UInt64.ZERO);
tekuNodeConfigBuilder.withDenebEpoch(UInt64.ZERO);
tekuNodeConfigBuilder.withElectraEpoch(UInt64.ZERO);
tekuNodeConfigBuilder.withFuluEpoch(UInt64.ZERO);| } | ||
|
|
||
| private Map<EngineApiMethod, EngineJsonRpcMethod<?>> fuluSupportedMethods() { | ||
| return new HashMap<>(); |
There was a problem hiding this comment.
Not sure on this, it means no supported methods for FULU. Are you sure?
There was a problem hiding this comment.
No. It means we are not adding any FULU specific methods.
| afterBeaconStateClass = BeaconStateElectra.class; | ||
| yield TestSpecFactory.createMinimalWithElectraForkEpoch(milestoneTransitionEpoch); | ||
| } | ||
| case FULU -> { |
There was a problem hiding this comment.
do we need to add FULU to TestSpecContext also?
| } | ||
|
|
||
| @Test | ||
| public void equals_denebConfigDiffer() { |
There was a problem hiding this comment.
I'd add shouldOverrideBlobRelatedValuesValues like in Electra test also
| } | ||
|
|
||
| @Test | ||
| public void equals_differentRandomValues() { |
There was a problem hiding this comment.
What we are testing is that the equals() method still works for different values, in this case, returning false.
| } | ||
|
|
||
| @Test | ||
| public void equals_denebConfigDiffer() { |
94f003d to
c83895c
Compare
zilm13
left a comment
There was a problem hiding this comment.
LGTM, just few nits
Also it may worth adding/updating here or in the separate PRs:
SpecConfigBuilderTest
SpecMilestoneTest
| NUMBER_OF_COLUMNS: 128 | ||
| NUMBER_OF_CUSTODY_GROUPS: 128 | ||
| DATA_COLUMN_SIDECAR_SUBNET_COUNT: 128 | ||
| MAX_REQUEST_DATA_COLUMN_SIDECARS: 512 |
There was a problem hiding this comment.
| NUMBER_OF_COLUMNS: 128 | ||
| NUMBER_OF_CUSTODY_GROUPS: 128 | ||
| DATA_COLUMN_SIDECAR_SUBNET_COUNT: 128 | ||
| MAX_REQUEST_DATA_COLUMN_SIDECARS: 512 |
There was a problem hiding this comment.
| NUMBER_OF_COLUMNS: 128 | ||
| NUMBER_OF_CUSTODY_GROUPS: 128 | ||
| DATA_COLUMN_SIDECAR_SUBNET_COUNT: 128 | ||
| MAX_REQUEST_DATA_COLUMN_SIDECARS: 512 |
There was a problem hiding this comment.
I'd use minimal config number (16384) here and in other non spec-ed configs
d39a816 to
8e46ba5
Compare
8e46ba5 to
20cb289
Compare
* Add read functionality to blobs archiving (#9318) * Schedule Gnosis Pectra hard-fork (#9340) * Updated ref tests v1.5.0-beta.5 (#9341) * Created Fulu skeleton (#9325) * Made checkpoint-state-url and initial-state mutually exclusive (#9342) It's hard to reason on how these two should interact, and logically they're mutually exclusive, so this just solidifies the theory that they shouldn't be used together. Signed-off-by: Paul Harris <paul.harris@consensys.net> * partial 3rd party updates and errorprone updates (#9351) I ended up disabling the instanceof errorprone, as it's very very common for us to be doing it apparently. I started changing things but in the interest of time will raise a ticket Signed-off-by: Paul Harris <paul.harris@consensys.net> * Make builder timeouts only for the HTTP call (#9353) Co-authored-by: Lucas Saldanha <lucas.saldanha@consensys.net> * more 3rd party updates (#9355) Signed-off-by: Paul Harris <paul.harris@consensys.net> * KZG updates from das branch (#9335) * Avoid combining validators by aggregating committee when using DVT (#9357) * builder json format, make media type more compatible (#9360) * refactored expected withdrawals and added test cases (#9361) Signed-off-by: Paul Harris <paul.harris@consensys.net> * Use `URI.create(...).toURL()` removing URL constructor deprecation (#9364) * clear-changelog (#9366) * New infra stream module (#9362) * feat: Add support for reading static peers from file (#9328) Co-authored-by: Paul Harris <paul.harris@consensys.net> * Start cleaning up `api/schema` (#9376) * Start cleaning up `api/schema` - moved `ValidatorStatus` - Removed `Version` - Moved `PublicKeyException` - Moved `EventType` Signed-off-by: Paul Harris <paul.harris@consensys.net> * cleanup the rest of api/schema (#9377) Signed-off-by: Paul Harris <paul.harris@consensys.net> * Added KZG computeCells method (#9375) * Added KZG computeCells method * Moved KZG no-operation logic into testFixture class * Fulu Schemas (ssz + datastructures) - including fulu ssz ref tests (#9363) * Added context to the sync committee duties error (#9380) Signed-off-by: Paul Harris <paul.harris@consensys.net> * Returning custody_group_count on node/identity Beacon API (#9381) * may 3rd party updates (#9388) Signed-off-by: Paul Harris <paul.harris@consensys.net> * Added snakeyaml to allow more complicated config reading (#9390) - added tests to show reading lists of objects (BPO related) - added a testcase to show that hex is read correctly - added testcase to show specConfigReader isnt filling defaults when we read local config files Partially addresses #9365 Signed-off-by: Paul Harris <paul.harris@consensys.net> * Add FULU to the Spec Factory (#9373) * Improve attestation bits aggregator electra (#9393) * added a DeserializableConfigTypeDefinition (#9396) - Tests demonstrate that this should at least be very close to what we require for reading BPO schedule. Partially addresses #9365 Signed-off-by: Paul Harris <paul.harris@consensys.net> * Updating ref test to 1.5.0 stable (#9398) * Adding DataColumnsByRootIdentifier container * PR comments * need more coffee... * uniforms the AggregatingAttestationPool interface (#9401) * Avoids potential mutability of the result of getCommitteeBits (#9402) --------- Signed-off-by: Paul Harris <paul.harris@consensys.net> Co-authored-by: Stefan Bratanov <stefan.bratanov93@gmail.com> Co-authored-by: Lucas Saldanha <lucas.saldanha@consensys.net> Co-authored-by: Paul Harris <paul.harris@consensys.net> Co-authored-by: Enrico Del Fante <enrico.delfante@consensys.net> Co-authored-by: crStiv <cryptostiv7@gmail.com> Co-authored-by: Mehdi AOUADI <mehdi.aouadi@consensys.net> Co-authored-by: Lucas Saldanha <lucascrsaldanha@gmail.com>
* Add read functionality to blobs archiving (#9318) * Schedule Gnosis Pectra hard-fork (#9340) * Updated ref tests v1.5.0-beta.5 (#9341) * Created Fulu skeleton (#9325) * Made checkpoint-state-url and initial-state mutually exclusive (#9342) It's hard to reason on how these two should interact, and logically they're mutually exclusive, so this just solidifies the theory that they shouldn't be used together. Signed-off-by: Paul Harris <paul.harris@consensys.net> * partial 3rd party updates and errorprone updates (#9351) I ended up disabling the instanceof errorprone, as it's very very common for us to be doing it apparently. I started changing things but in the interest of time will raise a ticket Signed-off-by: Paul Harris <paul.harris@consensys.net> * Make builder timeouts only for the HTTP call (#9353) Co-authored-by: Lucas Saldanha <lucas.saldanha@consensys.net> * more 3rd party updates (#9355) Signed-off-by: Paul Harris <paul.harris@consensys.net> * KZG updates from das branch (#9335) * Avoid combining validators by aggregating committee when using DVT (#9357) * builder json format, make media type more compatible (#9360) * refactored expected withdrawals and added test cases (#9361) Signed-off-by: Paul Harris <paul.harris@consensys.net> * Use `URI.create(...).toURL()` removing URL constructor deprecation (#9364) * clear-changelog (#9366) * New infra stream module (#9362) * feat: Add support for reading static peers from file (#9328) Co-authored-by: Paul Harris <paul.harris@consensys.net> * Start cleaning up `api/schema` (#9376) * Start cleaning up `api/schema` - moved `ValidatorStatus` - Removed `Version` - Moved `PublicKeyException` - Moved `EventType` Signed-off-by: Paul Harris <paul.harris@consensys.net> * cleanup the rest of api/schema (#9377) Signed-off-by: Paul Harris <paul.harris@consensys.net> * Added KZG computeCells method (#9375) * Added KZG computeCells method * Moved KZG no-operation logic into testFixture class * Fulu Schemas (ssz + datastructures) - including fulu ssz ref tests (#9363) * Added context to the sync committee duties error (#9380) Signed-off-by: Paul Harris <paul.harris@consensys.net> * Returning custody_group_count on node/identity Beacon API (#9381) * may 3rd party updates (#9388) Signed-off-by: Paul Harris <paul.harris@consensys.net> * Added snakeyaml to allow more complicated config reading (#9390) - added tests to show reading lists of objects (BPO related) - added a testcase to show that hex is read correctly - added testcase to show specConfigReader isnt filling defaults when we read local config files Partially addresses #9365 Signed-off-by: Paul Harris <paul.harris@consensys.net> * Add FULU to the Spec Factory (#9373) * Improve attestation bits aggregator electra (#9393) * added a DeserializableConfigTypeDefinition (#9396) - Tests demonstrate that this should at least be very close to what we require for reading BPO schedule. Partially addresses #9365 Signed-off-by: Paul Harris <paul.harris@consensys.net> * Updating ref test to 1.5.0 stable (#9398) * uniforms the AggregatingAttestationPool interface (#9401) * Avoids potential mutability of the result of getCommitteeBits (#9402) * Adding DataColumnsByRootIdentifier container (#9399) * Added an info message for the highest milestone (#9405) Also added a sanity check that will fail if the specFactory hasn't defined the highest milestone correctly, which has flow on effects. fixes #9400 Signed-off-by: Paul Harris <paul.harris@consensys.net> * Introduce `PooledAttestation` and `PooledAttestationWithData` (#9407) * Improve jdk 24 support and add docker-jdk24 (#9410) * add docker-jdk24 support * solves jdk 24 startup warning * rename AttestationBitsAggregator to AttestationBits (#9412) * Introduce aggregating pool interface (#9414) * [docker-jdk24] SIGHUP handler fix (#9413) * Optionally include validator indices in `PooledAttestation` (#9418) * Improve committeesSize retrieval in `AggregatingAttestationPool` (#9419) * Batch attestation duty scheduling for an epoch (#9374) * Updated MiscHelpersFulu + Added Gossip Logger (#9422) * Updated MiscHelpersFulu * Added GossipLogger and DAS related changes * gossip DAS related changes * fix test * fix runtime with noop fulu managers * bump MAX_SUBSCRIBED_TOPICS for Fulu and further forks * Use master's circle ci config * Remove duplicated workflows * Update test fixtures with random order changes * fix test * fix test * fix GetNewBlockV3Test * fix GetBlockTest * Gossip DAS related changes (#9423) * added bpo parsing to configuration (#9406) Signed-off-by: Paul Harris <paul.harris@consensys.net> * Fix ProduceBlockRequestTest for Fulu and updated order in random * Regenerate test fixtures with the changes in DataStructureUtil call order * Regenerate test fixtures with the changes in DataStructureUtil call order for GetNewBlockV3IntegrationTest * spotless * move GetDataColumnSidecars to tekuv1 space, fix openapi integration test * BPO - remove max blobs from fulu spec (#9427) Signed-off-by: Paul Harris <paul.harris@consensys.net> * Introduces `RewardBasedAttestationSorter` (#9428) * Fix integration tests after master upstream * Fix property tests + refactor some random datastructure utils to avoid fulu duplication * fix assemble * fix test * revert das test coverage changes * DAS storage changes (#9432) * Added config defaulting to the Config loader (#9439) Signed-off-by: Paul Harris <paul.harris@consensys.net> Co-authored-by: Gabriel Fukushima <gabrielfukushima@gmail.com> * Introduce `AttestationMatchingGroupV2` (#9438) * Fix DasSyncAcceptanceTest, fix DSL, remove broken DSL --------- Signed-off-by: Paul Harris <paul.harris@consensys.net> Co-authored-by: Stefan Bratanov <stefan.bratanov93@gmail.com> Co-authored-by: Lucas Saldanha <lucas.saldanha@consensys.net> Co-authored-by: Paul Harris <paul.harris@consensys.net> Co-authored-by: Enrico Del Fante <enrico.delfante@consensys.net> Co-authored-by: crStiv <cryptostiv7@gmail.com> Co-authored-by: Mehdi AOUADI <mehdi.aouadi@consensys.net> Co-authored-by: Lucas Saldanha <lucascrsaldanha@gmail.com> Co-authored-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
* Add read functionality to blobs archiving (#9318) * Schedule Gnosis Pectra hard-fork (#9340) * Updated ref tests v1.5.0-beta.5 (#9341) * Created Fulu skeleton (#9325) * Made checkpoint-state-url and initial-state mutually exclusive (#9342) It's hard to reason on how these two should interact, and logically they're mutually exclusive, so this just solidifies the theory that they shouldn't be used together. Signed-off-by: Paul Harris <paul.harris@consensys.net> * partial 3rd party updates and errorprone updates (#9351) I ended up disabling the instanceof errorprone, as it's very very common for us to be doing it apparently. I started changing things but in the interest of time will raise a ticket Signed-off-by: Paul Harris <paul.harris@consensys.net> * Make builder timeouts only for the HTTP call (#9353) Co-authored-by: Lucas Saldanha <lucas.saldanha@consensys.net> * more 3rd party updates (#9355) Signed-off-by: Paul Harris <paul.harris@consensys.net> * KZG updates from das branch (#9335) * Avoid combining validators by aggregating committee when using DVT (#9357) * builder json format, make media type more compatible (#9360) * refactored expected withdrawals and added test cases (#9361) Signed-off-by: Paul Harris <paul.harris@consensys.net> * Use `URI.create(...).toURL()` removing URL constructor deprecation (#9364) * clear-changelog (#9366) * New infra stream module (#9362) * feat: Add support for reading static peers from file (#9328) Co-authored-by: Paul Harris <paul.harris@consensys.net> * Start cleaning up `api/schema` (#9376) * Start cleaning up `api/schema` - moved `ValidatorStatus` - Removed `Version` - Moved `PublicKeyException` - Moved `EventType` Signed-off-by: Paul Harris <paul.harris@consensys.net> * cleanup the rest of api/schema (#9377) Signed-off-by: Paul Harris <paul.harris@consensys.net> * Added KZG computeCells method (#9375) * Added KZG computeCells method * Moved KZG no-operation logic into testFixture class * Fulu Schemas (ssz + datastructures) - including fulu ssz ref tests (#9363) * Added context to the sync committee duties error (#9380) Signed-off-by: Paul Harris <paul.harris@consensys.net> * Returning custody_group_count on node/identity Beacon API (#9381) * may 3rd party updates (#9388) Signed-off-by: Paul Harris <paul.harris@consensys.net> * Added snakeyaml to allow more complicated config reading (#9390) - added tests to show reading lists of objects (BPO related) - added a testcase to show that hex is read correctly - added testcase to show specConfigReader isnt filling defaults when we read local config files Partially addresses #9365 Signed-off-by: Paul Harris <paul.harris@consensys.net> * Add FULU to the Spec Factory (#9373) * Improve attestation bits aggregator electra (#9393) * added a DeserializableConfigTypeDefinition (#9396) - Tests demonstrate that this should at least be very close to what we require for reading BPO schedule. Partially addresses #9365 Signed-off-by: Paul Harris <paul.harris@consensys.net> * Updating ref test to 1.5.0 stable (#9398) * uniforms the AggregatingAttestationPool interface (#9401) * Avoids potential mutability of the result of getCommitteeBits (#9402) * Adding DataColumnsByRootIdentifier container (#9399) * Added an info message for the highest milestone (#9405) Also added a sanity check that will fail if the specFactory hasn't defined the highest milestone correctly, which has flow on effects. fixes #9400 Signed-off-by: Paul Harris <paul.harris@consensys.net> * Introduce `PooledAttestation` and `PooledAttestationWithData` (#9407) * Improve jdk 24 support and add docker-jdk24 (#9410) * add docker-jdk24 support * solves jdk 24 startup warning * rename AttestationBitsAggregator to AttestationBits (#9412) * Introduce aggregating pool interface (#9414) * [docker-jdk24] SIGHUP handler fix (#9413) * Optionally include validator indices in `PooledAttestation` (#9418) * Improve committeesSize retrieval in `AggregatingAttestationPool` (#9419) * Batch attestation duty scheduling for an epoch (#9374) * Updated MiscHelpersFulu + Added Gossip Logger (#9422) * Updated MiscHelpersFulu * Added GossipLogger and DAS related changes * Gossip DAS related changes (#9423) * added bpo parsing to configuration (#9406) Signed-off-by: Paul Harris <paul.harris@consensys.net> * BPO - remove max blobs from fulu spec (#9427) Signed-off-by: Paul Harris <paul.harris@consensys.net> * Introduces `RewardBasedAttestationSorter` (#9428) * DAS storage changes (#9432) * Added config defaulting to the Config loader (#9439) Signed-off-by: Paul Harris <paul.harris@consensys.net> Co-authored-by: Gabriel Fukushima <gabrielfukushima@gmail.com> * Introduce `AttestationMatchingGroupV2` (#9438) * Introduce `AggregatingAttestationPoolV2` (#9445) * Introduce `AggregatingAttestationPoolProfiler` implementations (#9447) Co-authored-by: Paul Harris <paul.harris@consensys.net> * PeerDAS RPC related changes (#9444) * Added a development flag to enable strict config loading (#9449) Strict loading was the default until yesterday, but it's caused a number of problems because if we push changes too fast etc it causes support issues. The new default permissive loading would mean that if a passed config file is missing an entry, we default it and print a warning. This new flag `--Xstartup-strict-config-loader-enabled` defaults to false, but if turned on will revert to failing on startup if a config being loaded doesn't include all of the expected configuration attributes. We could potentially start using this in AT to specify minimal changes to a configuration for a test, but for now I've left acceptance tests using strict loading. Signed-off-by: Paul Harris <paul.harris@consensys.net> * Updating reference tests to 1.6.0-alpha.0 (#9450) * Updating reference tests to 1.6.0-alpha.0 * Added Fulu networking tests * rename * Refactoring networking ATs * Add peer via api (#9431) * add initial impl of endpoint Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com> * add catch to cover address conversion failure Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com> * add tests for 200, 400 and 500 Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com> * spotless Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com> * add test for when discovery is not enabled Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com> * spotless Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com> * fix name of test after changing exception returned for invalid peer Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com> * add json def to paths Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com> * add changelog Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com> * PR review Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com> * spotless caught me again Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com> * change impl and tests to handle a single peer per api call Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com> * spotless Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com> --------- Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com> * `AggregatingAttestationPool` new CLI params (#9452) * poolV2-CLI * fix arity * getBits method names clearer (#9453) * Validate electra attestationData index received by BN (#9430) * fix integration test --------- Signed-off-by: Paul Harris <paul.harris@consensys.net> Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com> Co-authored-by: Stefan Bratanov <stefan.bratanov93@gmail.com> Co-authored-by: Lucas Saldanha <lucas.saldanha@consensys.net> Co-authored-by: Paul Harris <paul.harris@consensys.net> Co-authored-by: Enrico Del Fante <enrico.delfante@consensys.net> Co-authored-by: crStiv <cryptostiv7@gmail.com> Co-authored-by: Mehdi AOUADI <mehdi.aouadi@consensys.net> Co-authored-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
PR Description
Preparing for the upcome addition of all PeerDAS types and logic.
Fixed Issue(s)
N/A
Documentation
doc-change-requiredlabel to this PR if updates are required.Changelog