new eth tx manager #1484

tclemos · 2022-12-22T23:33:15Z

Closes #1464.

What does this PR do?

make eth tx manager generic for any tx

Reviewers

Main reviewers:

@ToniRamirezM
@arnaubennassar
@ARR552

…d9046

…5805bb

test/scripts/batchsender/main.go

…gas provider

…uencer and aggregator;

Dockerfile

ARR552 · 2023-01-16T12:38:01Z

cmd/approve.go

+
+	addrKeyStorePath := ctx.String(config.FlagKeyStorePath)
+	addrPassword := ctx.String(config.FlagPassword)
+


Why are you adding this as a param?? You are still reading the config file so I think the path and password are already in the configuration

The credentials were moved from [Etherman] to other components and there is no [Approve] section in the config file, that's why they are parameters.

ARR552 · 2023-01-16T12:41:43Z

cmd/run.go

+	ethTxManagerStorage, err := ethtxmanager.NewPostgresStorage(c.StateDB)
+	if err != nil {
+		log.Fatal(err)
+	}
+
+	etmStorage, err := ethtxmanager.NewPostgresStorage(c.StateDB)
+	if err != nil {
+		log.Fatal(err)
+	}


Why do you need to create it twice?

oops, copy/paste problems, fixing

ARR552 · 2023-01-16T14:29:18Z

etherman/etherman.go

-		sendSequencesOpts.Nonce = nonce
+	opts.NoSend = true
+	// force nonce, gas limit and gas price to avoid querying it from the chain
+	opts.Nonce = big.NewInt(1)


Why is set to 1. Shouldn't we set it to the current nonce? This transaction could fail due to nonce too low. Maybe I'm missing something

This method doesn't send any tx to the chain or process this in the EVM, we just use the methods in order to build the proper Txs fields taking advantage of the code generated from the SC.

Since we set the NoSend field we are safe to not get the tx sent to the chain, but we must set the fields Nonce, GasLimit, and GasPrice to a fixed value in order to avoid accessing the blockchain to try to estimate the gas or waste time and resources.

ARR552 · 2023-01-16T14:31:29Z

etherman/etherman.go

-	} else if etherMan.GasProviders.MultiGasProvider {
-		sendSequencesOpts.GasPrice = etherMan.getGasPrice(ctx)
+// BuildSequenceBatchesTxData builds a []bytes to be sent to the PoE SC method SequenceBatches.
+func (etherMan *Client) BuildSequenceBatchesTxData(sender common.Address, sequences []ethmanTypes.Sequence) (to *common.Address, value *big.Int, data []byte, err error) {


I dont understand why value is returned. Value will be always zero

I just wanted to keep these Build.... methods to return all the data needed to create a tx and the value field is one of them, do you like me to remove it?

As we are not going to need this value I think it is better to not have it

ARR552 · 2023-01-16T14:45:40Z

etherman/etherman.go

+// generateRandomAuth generates an authorization instance from a
+// randomly generated private key to be used to estimate gas for PoE
+// operations NOT restricted to the Trusted Sequencer
+func (etherMan *Client) generateRandomAuth() (bind.TransactOpts, error) {


I don't understand why this is required if we already know what is the sequencer and aggregator account

Well, in this new version of the EthTxManager, the Sequencer and Aggregator shouldn't know about the account secrets and only about the public address, the EthTxManager is the holder of the secrets and is the only one able to sign the Txs with it. Also, all the Transaction logic happens inside of the EthTxManager and they are not triggered by the Sequencer or Aggregator using Etherman.

With that said, Etherman should be used by the Sequencer and Aggregator in read-only mode and shouldn't require an authorization object, BUT the generated code to interact with the SC doesn't help much in terms of creating a Tx without signing it, because the methods are meant to send the TX to the blockchain, so in order to use these methods we must provide an authorization object.

That's why we have this method.

ToniRamirezM · 2023-01-17T14:42:57Z

aggregator/aggregator.go

@@ -460,12 +471,12 @@ func (a *Aggregator) unlockProofsToAggregate(ctx context.Context, proof1 *state.

 	if err != nil {
 		dbTx.Rollback(ctx) //nolint:errcheck


Maybe is good to log the rollback error. I would denote a DB access error.

ToniRamirezM · 2023-01-17T14:53:10Z

db/migrations/state/0005.sql

+    status     VARCHAR NOT NULL,
+    history    VARCHAR[],
+    block_num  BIGINT,
+    created_at TIMESTAMP NOT NULL,


In other tables we use TIMESTAMP WITH TIME ZONE. Maybe we should follow the same approach everywhere.

test/Makefile

sequencer/sequencesender.go

aggregator/aggregator.go

ethtxmanager/ethtxmanager.go

kind84

LGTM, impressive job @tclemos 👏

ToniRamirezM · 2023-01-19T08:21:38Z

db/migrations/state/0005.sql

@@ -0,0 +1,22 @@
+-- +migrate Down
+DROP TABLE IF EXISTS state.monitored_txs;


Maybe eth tx manager should have its own schema / database

we can't because we depend on reorgs and the database tx needs to be handled together to avoid out-of-sync data.

* Backport v0.0.1-RC4 & RC5 to develop (#1483) * update poe sc to security mechanisms version (#1467) * update poe sc to security mechanisms version * updated names, fix tests and remove ger timestamp and other fixes * fix rlp tool * fix linter * new genesis + fix * new smc * Update executor errors * Update executor errors * Update executor errors * Update executor errors * new l2 contract names * sequencewr unit test commented * fix test Co-authored-by: Alonso <[email protected]> Co-authored-by: ToniRamirezM <[email protected]> * Fix/1472 verify batches timeout (#1473) Closes #1472 ### What does this PR do? During testing we faced an issue of two consecutive verified batches on L1 without respecting the time spacing. The problem was that we were relying on the timer to check if we can start generating a new final proof. That timer is only reset after the final proof is sent to L1. This means that there's a time window between the generation of a final proof and its verification in which a new proof eligible to be final can be generated and immediately trigger another final proof generation. The fix introduced here is to have a `verifyingProof` flag that gets checked when the final proof generation process begins and released only when the verification phase ends (or in case of errors). Co-authored-by: Thiago Coimbra Lemos <[email protected]> Co-authored-by: Alonso <[email protected]> Co-authored-by: ToniRamirezM <[email protected]> * fix unpack the revert reason (#1475) Co-authored-by: tclemos <[email protected]> * fix: update the genesis file (#1489) * update the genesis * revert bytecode * upgrade git hub actions to the latest versions (#1471) * build(deps): bump github.com/go-git/go-billy/v5 from 5.3.1 to 5.4.0 (#1492) Bumps [github.com/go-git/go-billy/v5](https://github.com/go-git/go-billy) from 5.3.1 to 5.4.0. - [Release notes](https://github.com/go-git/go-billy/releases) - [Commits](go-git/go-billy@v5.3.1...v5.4.0) --- updated-dependencies: - dependency-name: github.com/go-git/go-billy/v5 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fatals changed to return error (#1494) * fatals changed to return error * comments * [Aggregator] Add metrics package and ConnectedProvers metric (#1502) * feat: add metrics pkg and connected provers metric This commit adds the `metrics` subpackage in the `aggregator` package. It also adds the connected provers gauge metric to keep track of the number of currently connected provers. Signed-off-by: Paolo Grisoli <[email protected]> * chore: expose aggregator metrics port This commit exposes the aggregator metrics port in `test/docker-compose.yml`. Signed-off-by: Paolo Grisoli <[email protected]> * feat: add WorkingProvers gauge metric (#1504) Closes #1503 ### What does this PR do? This PR adds the `aggregator_current_working_provers` gauge metric to keep track of the number of Provers that are busy generating a proof at any point in time. * build(deps): bump google.golang.org/grpc from 1.51.0 to 1.52.0 (#1525) Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.51.0 to 1.52.0. - [Release notes](https://github.com/grpc/grpc-go/releases) - [Commits](grpc/grpc-go@v1.51.0...v1.52.0) --- updated-dependencies: - dependency-name: google.golang.org/grpc dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): bump github.com/gorilla/websocket from 1.4.2 to 1.5.0 (#1510) Bumps [github.com/gorilla/websocket](https://github.com/gorilla/websocket) from 1.4.2 to 1.5.0. - [Release notes](https://github.com/gorilla/websocket/releases) - [Commits](gorilla/websocket@v1.4.2...v1.5.0) --- updated-dependencies: - dependency-name: github.com/gorilla/websocket dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): bump github.com/prometheus/common from 0.37.0 to 0.39.0 (#1511) Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.37.0 to 0.39.0. - [Release notes](https://github.com/prometheus/common/releases) - [Commits](prometheus/common@v0.37.0...v0.39.0) --- updated-dependencies: - dependency-name: github.com/prometheus/common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Feature/#1506 forced batch (#1523) * ForceBatch unit test * SequenceForceBatch unit test * Fix forcedBatch check * linter * Fix reset forcedBatch * batch and forcedbatch table modified * fix * Handle and log ROM OOC errors in executor (#1528) * Handle and log ROM OOC errors in executor * Fix IP * linter * improve test * solve conflicts * Feature/#1515 migration (#1531) * rpc and broadcast without migrations * mock * linter * fix unit test * migration flag for synchronizer and checkMigrations * run/stop aggregator command * discard invalid txs in the pool when checking for an underpriced tx (#1563) * allow array config params to be set via env vars (#1553) * new eth tx manager (#1484) * Fix: typos (#1567) * Fix: typo Fix: typo * Fix: typo Fix: typo * Fix: typo Fix: typo * Fix: typo Fix: typo * Fix: typo Fix: typo * Fix: typo Fix: typo * Fix: typo Fix: typo * Merge release/v0.0.1 into develop (#1561) Merge release/v0.0.1 changes into develop. The original release/v0.0.1 branch has been left intact to be able to perform hotfixes in case. * build(deps): bump github.com/spf13/viper from 1.14.0 to 1.15.0 (#1568) Bumps [github.com/spf13/viper](https://github.com/spf13/viper) from 1.14.0 to 1.15.0. - [Release notes](https://github.com/spf13/viper/releases) - [Commits](spf13/viper@v1.14.0...v1.15.0) --- updated-dependencies: - dependency-name: github.com/spf13/viper dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): bump github.com/urfave/cli/v2 from 2.23.7 to 2.24.1 (#1573) Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli) from 2.23.7 to 2.24.1. - [Release notes](https://github.com/urfave/cli/releases) - [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md) - [Commits](urfave/cli@v2.23.7...v2.24.1) --- updated-dependencies: - dependency-name: github.com/urfave/cli/v2 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Gas limit 150K + tests (#1551) * Gas limit 150K + tests * FreeClaimGasLimit as a config param * linter * fix config test * Gas limit 150K + tests (#1551) * Gas limit 150K + tests * FreeClaimGasLimit as a config param * linter * fix config test * Db flag to identify trusted verified batches (#1575) * Db flag to identify trusted verified batches * log * Feature/#1549 genesis txs (#1574) * genesis txs * linter * command removed * fixes * Feature/#1486 l2gasprice (#1548) * gasPricer as a component with the follower follower sets the l2gasprice depending on the l1 gas price * make command * Unit test + mocks * linter * fix * suggestions * suggestions * log message * suggesto to suggester + fix merge * file name fixed * jRPC - Batch methods and custom endpoint docs (#1554) * build(deps): bump google.golang.org/grpc from 1.52.0 to 1.52.3 (#1588) Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.52.0 to 1.52.3. - [Release notes](https://github.com/grpc/grpc-go/releases) - [Commits](grpc/grpc-go@v1.52.0...v1.52.3) --- updated-dependencies: - dependency-name: google.golang.org/grpc dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * New smc version (#1581) * New smc version * linter * fix * linter * genesis + network update * l2 claim gas limit * new aggregator wallet * config * add support to pre-EIP155 txs (#1582) * add support to pre-EIP155 txs * =) * naming things * add pre-EIP-155 tx e2e test * fix merge with develop * add check for the vitualized batch when testing pre-EIP155 tx * remove dependency of iden3 package from jrpc (#1596) * Merge * Feature/#1577 force (#1599) * forcedBatch Script * Sequence forcedBatch * linter * linter * linter * migrations * config * state.exit_root * state.exit_root * state.exit_root * state.exit_root * state.exit_root * state.exit_root * merge state migrations * merge state migrations * remerge with develop * Comment TestPreEIP155Tx * remove not used test --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Paolo Grisoli <[email protected]> Co-authored-by: Paolo Grisoli <[email protected]> Co-authored-by: Thiago Coimbra Lemos <[email protected]> Co-authored-by: Alonso <[email protected]> Co-authored-by: cool-developer <[email protected]> Co-authored-by: tclemos <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Alonso Rodriguez <[email protected]> Co-authored-by: omahs <[email protected]>

* WIP * wip: adding some implementation of the finalizer, defining needed interfaces by the finalizer from the dbmanager, and implementing the remaining resourses Sub. Signed-off-by: Nikolay Nedkov <[email protected]> * wip:2 Signed-off-by: Nikolay Nedkov <[email protected]> * fix: fixing comment Signed-off-by: Nikolay Nedkov <[email protected]> * wip: #4 Signed-off-by: Nikolay Nedkov <[email protected]> * fix: fixing bootstrap method in sequencer Signed-off-by: Nikolay Nedkov <[email protected]> * fix: fixing log in finalizer.reopenBatch Signed-off-by: Nikolay Nedkov <[email protected]> * DBManager WIP (#1512) * WIP * WIP * WIP * wip: adding implementation for .isCurrBatchAboveLimitWindow', and doing small refactor. Signed-off-by: Nikolay Nedkov <[email protected]> * fix: fixing handling of deadlines and events Signed-off-by: Nikolay Nedkov <[email protected]> * WIP: fixing logic. Signed-off-by: Nikolay Nedkov <[email protected]> * WIP: fixing logic. Signed-off-by: Nikolay Nedkov <[email protected]> * Synchronizer stores the accumulated input hash (#1535) * Initial worker implementation (still WIP) * merge Signed-off-by: Nikolay Nedkov <[email protected]> * Wip: merge Signed-off-by: Nikolay Nedkov <[email protected]> * merge Signed-off-by: Nikolay Nedkov <[email protected]> * Feature/dbmanager (#1524) * wip: adding implementation for .isCurrBatchAboveLimitWindow', and doing small refactor. Signed-off-by: Nikolay Nedkov <[email protected]> * WIP * WIP * WIP * WIP * WIP * WIP * WIP * WIP * ForceBatch unit test * SequenceForceBatch unit test * Fix forcedBatch check * linter * Fix reset forcedBatch * batch and forcedbatch table modified * fix * WIP * Finish dbmanager * Branches Integration Signed-off-by: Nikolay Nedkov <[email protected]> Co-authored-by: Nikolay Nedkov <[email protected]> Co-authored-by: Alonso <[email protected]> * implement last methods (#1542) * added getLastStateRoot (#1543) * fix,tests: fixing some methods and adding unit tests. Signed-off-by: Nikolay Nedkov <[email protected]> * Closing Signals Manager first implementation (#1550) * Closing Signals Manager first implementation * Closing Signals Manager first implementation * Closing Signals Manager first implementation * refactor * Added worker unit testing. Missing TODOs implementation done * Cahnge getLastGer implementation (#1556) * wip fixes Signed-off-by: Nikolay Nedkov <[email protected]> * wip fixes Signed-off-by: Nikolay Nedkov <[email protected]> * wip fixes Signed-off-by: Nikolay Nedkov <[email protected]> * fixed addrqueue.addTx returning wrong (newReadyTx, prevReadyTx) when the added tx was a notReadyTxs * Fix encoding bug in AddForcedBatch (#1544) The forced_batch table expects the raw_txs_data column to be a string (specifically a hex encoding of the raw transaction bytes). The corresponding GetForcedBatch function uses hex.DecodeString to turn the value of this column back into raw bytes. However, AddForcedBatch is missing a hex.EncodeToString, so it is actually interpreting the raw transaction bytes as an ASCII representation of a hex-encoded string. * Feature/executor error (#1566) * Handle executor and rom erros from executor * convert touched addresses * add debug logs * add debug logs * store request in case of executor error * store request in case of executor error * minor fixes * WIP * convert readwriteaddresses to map * add test * add method to mark wip txs as pending Co-authored-by: ToniRamirezM <[email protected]> * fix: wip fix of prepareProcessRequestFromState to fetch txs when reprocess is done. Signed-off-by: Nikolay Nedkov <[email protected]> * test (#1569) * fix: fixing benchmark test and several bugs. Signed-off-by: Nikolay Nedkov <[email protected]> * Fix linter and tests (#1576) * fix linter * fix linter * fix linter * fix linter * fix linter * fix linter * fix linter * fix linter * fix linter * fix linter * fix linter * fix linter * fix linter * fix linter * fix linter * fix linter * fix linter * fix linter * fix linter * fix linter * fix linter * fix linter * remove testMain (#1585) * Fix errors found while running bridge/GER tests * update mock files * Add forced batch closing signal unit test (#1593) * add forced batch closing signal unit test * add forced batch closing signal unit test * add forced batch closing signal unit test * add forced batch closing signal unit test * test: adding sequencer benchmark test with ERC20 transfers. Signed-off-by: Nikolay Nedkov <[email protected]> * adding erc-20 benchmark test, adding metrics for wokerker processing time and fixing unit tests of finalizer Signed-off-by: Nikolay Nedkov <[email protected]> * adding missing mutex lock in woker.MoveTxToNotReady Signed-off-by: Nikolay Nedkov <[email protected]> * fix: fixing benchmark result print messages. Signed-off-by: Nikolay Nedkov <[email protected]> * [New Sequencer] Merge develop into new sequencer branch (#1602) * Backport v0.0.1-RC4 & RC5 to develop (#1483) * update poe sc to security mechanisms version (#1467) * update poe sc to security mechanisms version * updated names, fix tests and remove ger timestamp and other fixes * fix rlp tool * fix linter * new genesis + fix * new smc * Update executor errors * Update executor errors * Update executor errors * Update executor errors * new l2 contract names * sequencewr unit test commented * fix test Co-authored-by: Alonso <[email protected]> Co-authored-by: ToniRamirezM <[email protected]> * Fix/1472 verify batches timeout (#1473) Closes #1472 ### What does this PR do? During testing we faced an issue of two consecutive verified batches on L1 without respecting the time spacing. The problem was that we were relying on the timer to check if we can start generating a new final proof. That timer is only reset after the final proof is sent to L1. This means that there's a time window between the generation of a final proof and its verification in which a new proof eligible to be final can be generated and immediately trigger another final proof generation. The fix introduced here is to have a `verifyingProof` flag that gets checked when the final proof generation process begins and released only when the verification phase ends (or in case of errors). Co-authored-by: Thiago Coimbra Lemos <[email protected]> Co-authored-by: Alonso <[email protected]> Co-authored-by: ToniRamirezM <[email protected]> * fix unpack the revert reason (#1475) Co-authored-by: tclemos <[email protected]> * fix: update the genesis file (#1489) * update the genesis * revert bytecode * upgrade git hub actions to the latest versions (#1471) * build(deps): bump github.com/go-git/go-billy/v5 from 5.3.1 to 5.4.0 (#1492) Bumps [github.com/go-git/go-billy/v5](https://github.com/go-git/go-billy) from 5.3.1 to 5.4.0. - [Release notes](https://github.com/go-git/go-billy/releases) - [Commits](go-git/go-billy@v5.3.1...v5.4.0) --- updated-dependencies: - dependency-name: github.com/go-git/go-billy/v5 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fatals changed to return error (#1494) * fatals changed to return error * comments * [Aggregator] Add metrics package and ConnectedProvers metric (#1502) * feat: add metrics pkg and connected provers metric This commit adds the `metrics` subpackage in the `aggregator` package. It also adds the connected provers gauge metric to keep track of the number of currently connected provers. Signed-off-by: Paolo Grisoli <[email protected]> * chore: expose aggregator metrics port This commit exposes the aggregator metrics port in `test/docker-compose.yml`. Signed-off-by: Paolo Grisoli <[email protected]> * feat: add WorkingProvers gauge metric (#1504) Closes #1503 ### What does this PR do? This PR adds the `aggregator_current_working_provers` gauge metric to keep track of the number of Provers that are busy generating a proof at any point in time. * build(deps): bump google.golang.org/grpc from 1.51.0 to 1.52.0 (#1525) Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.51.0 to 1.52.0. - [Release notes](https://github.com/grpc/grpc-go/releases) - [Commits](grpc/grpc-go@v1.51.0...v1.52.0) --- updated-dependencies: - dependency-name: google.golang.org/grpc dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): bump github.com/gorilla/websocket from 1.4.2 to 1.5.0 (#1510) Bumps [github.com/gorilla/websocket](https://github.com/gorilla/websocket) from 1.4.2 to 1.5.0. - [Release notes](https://github.com/gorilla/websocket/releases) - [Commits](gorilla/websocket@v1.4.2...v1.5.0) --- updated-dependencies: - dependency-name: github.com/gorilla/websocket dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): bump github.com/prometheus/common from 0.37.0 to 0.39.0 (#1511) Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.37.0 to 0.39.0. - [Release notes](https://github.com/prometheus/common/releases) - [Commits](prometheus/common@v0.37.0...v0.39.0) --- updated-dependencies: - dependency-name: github.com/prometheus/common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Feature/#1506 forced batch (#1523) * ForceBatch unit test * SequenceForceBatch unit test * Fix forcedBatch check * linter * Fix reset forcedBatch * batch and forcedbatch table modified * fix * Handle and log ROM OOC errors in executor (#1528) * Handle and log ROM OOC errors in executor * Fix IP * linter * improve test * solve conflicts * Feature/#1515 migration (#1531) * rpc and broadcast without migrations * mock * linter * fix unit test * migration flag for synchronizer and checkMigrations * run/stop aggregator command * discard invalid txs in the pool when checking for an underpriced tx (#1563) * allow array config params to be set via env vars (#1553) * new eth tx manager (#1484) * Fix: typos (#1567) * Fix: typo Fix: typo * Fix: typo Fix: typo * Fix: typo Fix: typo * Fix: typo Fix: typo * Fix: typo Fix: typo * Fix: typo Fix: typo * Fix: typo Fix: typo * Merge release/v0.0.1 into develop (#1561) Merge release/v0.0.1 changes into develop. The original release/v0.0.1 branch has been left intact to be able to perform hotfixes in case. * build(deps): bump github.com/spf13/viper from 1.14.0 to 1.15.0 (#1568) Bumps [github.com/spf13/viper](https://github.com/spf13/viper) from 1.14.0 to 1.15.0. - [Release notes](https://github.com/spf13/viper/releases) - [Commits](spf13/viper@v1.14.0...v1.15.0) --- updated-dependencies: - dependency-name: github.com/spf13/viper dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): bump github.com/urfave/cli/v2 from 2.23.7 to 2.24.1 (#1573) Bumps [github.com/urfave/cli/v2](https://github.com/urfave/cli) from 2.23.7 to 2.24.1. - [Release notes](https://github.com/urfave/cli/releases) - [Changelog](https://github.com/urfave/cli/blob/main/docs/CHANGELOG.md) - [Commits](urfave/cli@v2.23.7...v2.24.1) --- updated-dependencies: - dependency-name: github.com/urfave/cli/v2 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Gas limit 150K + tests (#1551) * Gas limit 150K + tests * FreeClaimGasLimit as a config param * linter * fix config test * Gas limit 150K + tests (#1551) * Gas limit 150K + tests * FreeClaimGasLimit as a config param * linter * fix config test * Db flag to identify trusted verified batches (#1575) * Db flag to identify trusted verified batches * log * Feature/#1549 genesis txs (#1574) * genesis txs * linter * command removed * fixes * Feature/#1486 l2gasprice (#1548) * gasPricer as a component with the follower follower sets the l2gasprice depending on the l1 gas price * make command * Unit test + mocks * linter * fix * suggestions * suggestions * log message * suggesto to suggester + fix merge * file name fixed * jRPC - Batch methods and custom endpoint docs (#1554) * build(deps): bump google.golang.org/grpc from 1.52.0 to 1.52.3 (#1588) Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.52.0 to 1.52.3. - [Release notes](https://github.com/grpc/grpc-go/releases) - [Commits](grpc/grpc-go@v1.52.0...v1.52.3) --- updated-dependencies: - dependency-name: google.golang.org/grpc dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * New smc version (#1581) * New smc version * linter * fix * linter * genesis + network update * l2 claim gas limit * new aggregator wallet * config * add support to pre-EIP155 txs (#1582) * add support to pre-EIP155 txs * =) * naming things * add pre-EIP-155 tx e2e test * fix merge with develop * add check for the vitualized batch when testing pre-EIP155 tx * remove dependency of iden3 package from jrpc (#1596) * Merge * Feature/#1577 force (#1599) * forcedBatch Script * Sequence forcedBatch * linter * linter * linter * migrations * config * state.exit_root * state.exit_root * state.exit_root * state.exit_root * state.exit_root * state.exit_root * merge state migrations * merge state migrations * remerge with develop * Comment TestPreEIP155Tx * remove not used test --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Paolo Grisoli <[email protected]> Co-authored-by: Paolo Grisoli <[email protected]> Co-authored-by: Thiago Coimbra Lemos <[email protected]> Co-authored-by: Alonso <[email protected]> Co-authored-by: cool-developer <[email protected]> Co-authored-by: tclemos <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Alonso Rodriguez <[email protected]> Co-authored-by: omahs <[email protected]> * adding missing mutex lock in woker.MoveTxToNotReady Signed-off-by: Nikolay Nedkov <[email protected]> * fix: fixing benchmark result print messages. Signed-off-by: Nikolay Nedkov <[email protected]> * adding missing part for processing forced batches, new http port exposing profiling endpoints when enabled, fixing lastGER hash used on opening new batch, and adding context to 'finalizer.processTransation' logs. * fixing lint error Signed-off-by: Nikolay Nedkov <[email protected]> * adding missing port exposing for profilintin test/docker-compose.yml Signed-off-by: Nikolay Nedkov <[email protected]> * adding prints for fetching methods in benchmark/.../metrics.go Signed-off-by: Nikolay Nedkov <[email protected]> * Fix closing signals (#1623) * adding wip fixes and the closing signal emit from the sequencesender (#1625) Signed-off-by: Nikolay Nedkov <[email protected]> * fix forced batch query (#1630) * L1Timeout (#1635) * L1Timeout * L1Timeout * L1Timeout * fix config files * wip * fix config files * fix config files * fix test * fix test * wip: fixing forced batches. Signed-off-by: Nikolay Nedkov <[email protected]> * decode txs * wip Signed-off-by: Nikolay Nedkov <[email protected]> * wip #2 Signed-off-by: Nikolay Nedkov <[email protected]> * wip #3 Signed-off-by: Nikolay Nedkov <[email protected]> * wip fixes and improvements Signed-off-by: Nikolay Nedkov <[email protected]> * wip making print of reprocess state root difference to not be Fatal. Signed-off-by: Nikolay Nedkov <[email protected]> * wip: removing redundant fatal in reprocessBatch and fixing unit tests. Signed-off-by: Nikolay Nedkov <[email protected]> * wip: removing redundant fatal Signed-off-by: Nikolay Nedkov <[email protected]> * wip: fixing to update StateRoot when passing no txs for processing to close empty batch. Signed-off-by: Nikolay Nedkov <[email protected]> * wip fixes Signed-off-by: Nikolay Nedkov <[email protected]> * wip updating config files. Signed-off-by: Nikolay Nedkov <[email protected]> * changing when checking nextSendingToL1Deadline if there are no txs in the batch not to close the batch but to update the deadline Signed-off-by: Nikolay Nedkov <[email protected]> * removing debug prints Signed-off-by: Nikolay Nedkov <[email protected]> * wip Signed-off-by: Nikolay Nedkov <[email protected]> * update executor image to e7ac5c4 * smc update * GetForks + test * integrate getForks * l1 network docker image * linter * Fee recipient * genesis * linter * fix unit test * coinbase * claim signature * smc update l2coinbase * remove genesis txs * genesis * genesis docker + new addresses and names * addresses * genesis fix * fix script * fix GenBlockNum * fix docker-compose * fix prover config * update makefile * fix config toml files * fix db schema * timestamp * fix state tests * minor fixes * fix migrations test * update prover * prover image * prover image * fix makefile * fix fork_id in ProcessBatch * fix fork_id in ProcessBatch * update prover image * update test * update test * do not update MT in reprocess * change config * update MT in reprocess * fix RlpFieldsToLegacyTx * fix RlpFieldsToLegacyTx * fix RlpFieldsToLegacyTx * change config * change config * increase test timeout * increase test timeout * wip fixes Signed-off-by: Nikolay Nedkov <[email protected]> * check fix Signed-off-by: Nikolay Nedkov <[email protected]> * fixing 'debug.node.config.toml' Signed-off-by: Nikolay Nedkov <[email protected]> * removing unused config props for sequencer Signed-off-by: Nikolay Nedkov <[email protected]> --------- Signed-off-by: Nikolay Nedkov <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Paolo Grisoli <[email protected]> Co-authored-by: Nikolay Nedkov <[email protected]> Co-authored-by: Alonso Rodriguez <[email protected]> Co-authored-by: agnusmor <[email protected]> Co-authored-by: agnusmor <[email protected]> Co-authored-by: Alonso <[email protected]> Co-authored-by: Jeb Bearer <[email protected]> Co-authored-by: Paolo Grisoli <[email protected]> Co-authored-by: Thiago Coimbra Lemos <[email protected]> Co-authored-by: cool-developer <[email protected]> Co-authored-by: tclemos <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: omahs <[email protected]>

tclemos added 8 commits December 22, 2022 20:05

cherry-pick refactoring eth tx manager for sequence batches 793b684

4495d35

refactoring eth tx manager for sequence batches

04de9db

add wait sync for sequences

b0087f0

cherry-pick implement verify batch for new eth tx manager 98767d1

0314a2f

add ci checks for release branches

fd9cae1

cherry-pick fix returning error 6d5dc97

0175494

cherry-pick eth tx manager initial hard persistence implementation 51…

a10bdd0

…d9046

cherry-pick refactoring eth tx manager to make it generic to any tx 6…

b0681de

…5805bb

cla-bot bot added the cla-signed label Dec 22, 2022

tclemos self-assigned this Dec 22, 2022

tclemos mentioned this pull request Dec 22, 2022

Fix/eth tx manager refactoring #1462

Closed

tclemos added 5 commits December 23, 2022 11:14

Merge branch 'develop' into feature/develop/new-eth-tx-manager

7f64e46

partial eth tx manager postgres implementation

ccc8bf3

replace memory storage by postgresql for eth tx manager

c45fa63

mock etherman for eth tx manager tests and add more tests

e86969f

integrate sequencer with new eth tx manager; adds owner concept

8da0733

kind84 mentioned this pull request Jan 4, 2023

[Aggregator] Switch to new ethtxmanager interface #1507

Closed

tclemos added 7 commits January 5, 2023 16:12

add reorg to eth tx manager; add batch number to sequences

2203135

eth-tx-manager is now a component

5149c78

fix eth tx manager config and config tests

4b4ba66

test improvements and fix test race condition

54c40e2

fix linter issues

e26a950

Merge branch 'develop' into feature/develop/new-eth-tx-manager

235cd22

integrate aggregator with new eth tx manager

99703d2

tclemos commented Jan 7, 2023

View reviewed changes

test/scripts/batchsender/main.go Outdated Show resolved Hide resolved

tclemos added 5 commits January 9, 2023 08:13

fix etherman public suggested gas price method to use internal multi …

d5e90e5

…gas provider

fix e2e tests

15ebb8a

review aggregator ethTxManager integration; diff private keys for seq…

5832aaa

…uencer and aggregator;

fix linter issues

434ee36

improvements on eth tx manager for failed txs

c69c63f

tclemos added 6 commits January 11, 2023 12:28

fix unit tests

afcfbf5

fix e2e tests

18740fa

fix migrations

c4b7b25

Merge branch 'develop' into feature/develop/new-eth-tx-manager

38192f5

fix eth tx manager tx review

c160a17

add unit tests for execution reverted

78076c7

tclemos requested review from ARR552, kind84, arnaubennassar and ToniRamirezM January 13, 2023 14:25

tclemos marked this pull request as ready for review January 13, 2023 14:26

tclemos requested a review from cool-develope as a code owner January 13, 2023 14:26

add logs to eth-tx-manager reorg

7bdfceb

ARR552 reviewed Jan 16, 2023

View reviewed changes

tclemos added 2 commits January 16, 2023 13:29

removing unnecessary code

16c448d

Merge branch 'develop' into feature/develop/new-eth-tx-manager

9602f15

ToniRamirezM reviewed Jan 17, 2023

View reviewed changes

tclemos added 3 commits January 17, 2023 15:33

remove value from etherman Build...TxData methods

d207846

handle rollback error

0bb30d8

add WITH TIME ZONE to timestamps to follow the same approach everywhere

fa19a07

kind84 reviewed Jan 18, 2023

View reviewed changes

fix PR suggestions

779d7ba

kind84 approved these changes Jan 18, 2023

View reviewed changes

ARR552 approved these changes Jan 19, 2023

View reviewed changes

ToniRamirezM approved these changes Jan 19, 2023

View reviewed changes

tclemos merged commit 40f340b into develop Jan 19, 2023

tclemos deleted the feature/develop/new-eth-tx-manager branch January 19, 2023 14:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

new eth tx manager #1484

new eth tx manager #1484

tclemos commented Dec 22, 2022

ARR552 Jan 16, 2023

tclemos Jan 16, 2023

ARR552 Jan 16, 2023

tclemos Jan 16, 2023

ARR552 Jan 16, 2023

tclemos Jan 16, 2023

ARR552 Jan 16, 2023

tclemos Jan 16, 2023

ARR552 Jan 17, 2023

ARR552 Jan 16, 2023

tclemos Jan 16, 2023

ToniRamirezM Jan 17, 2023

ToniRamirezM Jan 17, 2023

kind84 left a comment

ToniRamirezM Jan 19, 2023

tclemos Jan 19, 2023


		addrKeyStorePath := ctx.String(config.FlagKeyStorePath)
		addrPassword := ctx.String(config.FlagPassword)

		@@ -460,12 +471,12 @@ func (a Aggregator) unlockProofsToAggregate(ctx context.Context, proof1 state.

		if err != nil {
		dbTx.Rollback(ctx) //nolint:errcheck

		@@ -0,0 +1,22 @@
		-- +migrate Down
		DROP TABLE IF EXISTS state.monitored_txs;

new eth tx manager #1484

new eth tx manager #1484

Conversation

tclemos commented Dec 22, 2022

What does this PR do?

Reviewers

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kind84 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment