feat: remove txs which do not end up in a block out of the mempool#15
Merged
bharath-123 merged 4 commits intomainfrom May 15, 2024
Merged
feat: remove txs which do not end up in a block out of the mempool#15bharath-123 merged 4 commits intomainfrom
bharath-123 merged 4 commits intomainfrom
Conversation
bharath-123
force-pushed
the
bharath/remove-invalid-txs
branch
from
bbefcc6 to
e30e123
Compare
bharath-123
commented
May 9, 2024
| } | ||
| start := time.Now() | ||
| full := w.getSealingBlock(emptyParams) | ||
| full := w.getSealingBlock(fullParams) |
Contributor
Author
There was a problem hiding this comment.
In vanilla geth, an empty block is created and then a full block is created. This is mainly to spend more time to optimize for gas fees and also to return at least an empty block if we were not able to create a full block on time.
We don't need this astria since we just blindly use the txs received from ExecuteBlock. so fullParams makes more sense than emptyParams over here.
bharath-123
force-pushed
the
bharath/remove-invalid-txs
branch
2 times, most recently
from
c8f6554 to
ad9cced
Compare
bharath-123
force-pushed
the
bharath/remove-invalid-txs
branch
from
ad9cced to
fe45f4e
Compare
bharath-123
requested review from
joroshiba,
mycodecrafting and
noot
as code owners
Member
|
We’ve had previous discussions about the management of txs between the geth/rollup mempool and the Astria sequencer mempool. Is the focus of this PR to remove transactions which were paid to be included in Astria but didn’t get executed in geth because we ran into the execution gas limit? |
bharath-123
commented
May 9, 2024
miner/worker.go
Outdated
| } | ||
| if err != nil { | ||
| log.Trace("Marking transaction as invalid", "hash", tx.Hash(), "err", err) | ||
| w.eth.TxPool().UpdateAstriaInvalid(tx) |
Contributor
Author
There was a problem hiding this comment.
should we also remove if a tx is skipped because of a high nonce or low nonce? or only because it exceeded the gas limit?
Contributor
Author
There was a problem hiding this comment.
We ideally should to maintain 1:1 block determinism. So we should include all of these cases.
joroshiba
approved these changes
May 13, 2024
core/txpool/blobpool/blobpool.go
Outdated
| func (p *BlobPool) SetAstriaOrdered(types.Transactions) {} | ||
| func (p *BlobPool) ClearAstriaOrdered() {} | ||
| func (p *BlobPool) UpdateAstriaExcludedFromBlock(*types.Transaction) {} | ||
| func (p *BlobPool) AstriaExcludedFromBlock() *types.Transactions { return &types.Transactions{} } |
Member
There was a problem hiding this comment.
Seems like this public function is added solely for test purposes? Is there another way to manage that?
Contributor
Author
There was a problem hiding this comment.
don't think there is, we can't attribute functions with a #cfg[test] like how we do in rust :( , we need to do hacks like https://github.com/golang/go/blob/master/src/reflect/export_test.go
There are some cases in the Geth codebase where they have written public functions only for testing like this one:
astria-geth/miner/payload_building.go
Line 144 in a9326e1
core/txpool/blobpool/blobpool.go
Outdated
| func (p *BlobPool) AstriaOrdered() *types.Transactions { return &types.Transactions{} } | ||
| func (p *BlobPool) SetAstriaOrdered(types.Transactions) {} | ||
| func (p *BlobPool) ClearAstriaOrdered() {} | ||
| func (p *BlobPool) UpdateAstriaExcludedFromBlock(*types.Transaction) {} |
Member
There was a problem hiding this comment.
nit on naming... perhaps NewAstriaOrderedExcludedTx? naming here is always going to be awkward as it's actually a method on a subtype, its semantically kinda wrong.
Not for this PR, but the adding stuff to the mempool has always felt a bit awkward. Might makes more sense to add payload argument or something, or even create a new subpool.
bharath-123
force-pushed
the
bharath/remove-invalid-txs
branch
from
d2fc307 to
67d9f9e
Compare
Contributor
Author
|
Merging this to rebase on top of the PR to fix tests |
bharath-123
added a commit
that referenced
this pull request
May 29, 2024
When the txs we receive from `ExecuteBlocks` call fail to end up in a block, we keep them lying around the geth mempool but clear it from the astria mempool. This PR attempts to remove such txs from the geth mempool.
SuperFluffy
pushed a commit
that referenced
this pull request
Apr 28, 2025
3d160b2 is the last commit before the infamous squash + merge of geth into astria-geth The commit messages below are left unchanged for now so that they can be cleaned up later. Feature/grpc execution api (#1) * add buf config and generated grpc code * poc e2e grpc communication * comment out panicking code * logging help * now reads cli args. now stops grpc server on shutdown. * add mutex to GRPCServerHandler update readme containerize w/ github action to build and push to ghcr (#3) fix gh action syntax (#4) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax build and push manually (#5) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax * dont use github action, do it ourselves * push manually Feature/containerize (#6) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax * dont use github action, do it ourselves * push manually * correct multi line update SubmitTransaction to send tx to metro ethclient: ensure returned subscription is nil on error (#26976) core/state, trie: remove Try prefix in Trie accessors (#26975) This change renames StateTrie methods to remove the Try* prefix. We added the Trie methods with prefix 'Try' a long time ago, working around the problem that most existing methods of Trie did not return the database error. This weird naming convention has persisted until now. Co-authored-by: Gary Rong <garyrong0905@gmail.com> metrics/librato: ensure resp.body closed (#26969) This change ensures that we call Close on a http response body, in various places in the source code (mostly tests) core/vm: use atomic.Bool (#26951) Make use of new atomic types --------- Co-authored-by: Felix Lange <fjl@twurst.com> Co-authored-by: Martin Holst Swende <martin@swende.se> core/bloombits: use atomic type (#26993) core/state: use atomic.Bool (#26992) graphql: fix data races (#26965) Fixes multiple data races caused by the fact that resolving fields are done concurrently by the graphql library. It also enforces caching at the stateobject level for account fields. eth/tracers/native: prevent panic for LOG edge-cases (#26848) This PR fixes OOM panic in the callTracer as well as panicing on opcode validation errors (e.g. stack underflow) in callTracer and prestateTracer. Co-authored-by: Martin Holst Swende <martin@swende.se> internal/debug: add log.logfmt flag to set logging to use logfmt (#26970) docs: update outdated DeriveSha docs comment (#26968) add localnet genesis update metro-transactions dep Adding features to mempool to support our pre-ordered txs (#2) * Adding features to mempool to support our pre-ordered txs * Add framing for gRPC to execute blocks * Remove public engine API call * Fix circular dependencies * Updated to use new Init, and fill out starting attributes * Add clear astriaordered, cleanup * readme fix * add bash and jq to final docker image * make txpool interface * no more panics, update DoBlock to also update state + store block * cleanup * set post-merge at genesis * cleanup * doc update * remove txpool interface changes * cleanup * cleanup * build and push images wih tags defined by git tags * build for multiple architectures. use docker-metadata action for semver * add push: true * only build arm for git tags/releases --------- Co-authored-by: Jesse Snyder <jessetsnyder@gmail.com> Co-authored-by: elizabeth <elizabethjbinks@gmail.com> tag image with latest for builds from astria branch (#8) rename state_root to block_hash use FROM --platform=$BUILDPLATFORM to build arm images correctly (#9) use v4 of build and push action (#10) * use FROM --platform=$BUILDPLATFORM to build arm images correctly * use v4 of docker build and push fix arm builds. only build arm for tags and merges to default branch. (#12) * fix arm builds. only build arm for tags and merges to default branch. * fix gh action function syntax * more correct comment * build for semver tags Features and fixes needed for contract deployment test (#13) * add and remove tx from geth mempool so forge can deploy a contract * pass metro addr and port by flag * fixes from pr feedback * formatting implement FinalizeBlock gRPC call bump metro-transactions dep Update Protos (#16) * uses new protos * remove old protos update deps update deps no more unknown/unknown image (#19) * no more unknown/unknown image * hardcode condition Changing from prev_state_root to prev_block_hash remove metro integrate submission to sequencer log updates use env vars for chain id/tendermint endpoint rename to cometbft Make the environment variables actually work cleaner simple logging change removed sleep before getPayload by removing goroutine in buildPayload removed unneeded comment removed for loop from buildPayload Remove direct sequencer submission cleanup go.mod Migrate to the v1alpha1 API Missed pieces set genesis block as head/safe/final remove all mentions of metro Initial implementation of Execution v1alpha2 api Integrate updates remove need for consensus api Updates small updates Update grpc/execution/server.go Co-authored-by: noot <36753753+noot@users.noreply.github.com> logging remove unused dependency Mark the chain 'finalized' on startup updates txpool fixes, remove v1alpha1, use forkchoice some cleanup formatting and such cleanup think it's good add new fields to base example genesis Update gh action for release branch Add timestamp to block generated from header Add missing parent hash chore: add some metrics specifically around astria tx execution small refactor fix feat: everything needed for genesis fix review + metrics update to prod pushed buf fix after grpc update switch to new execution apis package feat: support new API utilizing rollup data update to latest proto update docker to login implement DepositTx type and handle in state transition and execute_block cleanup fixes to get deposits working, update genesis.json cleanup cleanup exit node if genesis astria fields are unset add bridge address and asset IDs to genesis and check during execution don't allow deposit or blob txs in sequenced data minimum changes required for fork (#2) Updates the codeowners and sets a new package repo for `astria-geth` repo. Once this is merged, I can push the astria specific tags and we will get built images for old tags on this repo. feat: update protos, config for fees, updated bridging (#1) Allow setting eip 1559 fee params (#3) Adds a per block height config to set some EIP1559 fee params (elasticity and denominator) + adds a min base fee parameter. collect the base fee instead of burning it (#5) EIP-1559 burns the base fee. This doesn't make sense for rollups. The set fee recipient should collect it. Chore: Add forkdiff gh-page (#14) we need a static webpage to show changes made to astria-geth fork, this should be used both internally and externally. using a [forkdiff](https://github.com/protolambda/forkdiff) tool to create a static webpage. add a fork.yaml file structs the changes view and define the commits to compare with pages.yaml workflow deploys to gh-pages Currently deploying on PR for testing. feat: remove txs which do not end up in a block out of the mempool (#15) When the txs we receive from `ExecuteBlocks` call fail to end up in a block, we keep them lying around the geth mempool but clear it from the astria mempool. This PR attempts to remove such txs from the geth mempool. chore: make server logs less noisy (#12) Moves the initial request logs to debug logs, reduces the amount of information printed, and ensures okay formatting. Logs from Geth were very messy, block hashes were messy bytes, and all the information from each request was logged out unnecessarily. fix: continue the tx validation loop if assetId length check fails (#18) Currently, we check if the length of the assetId sent by the rollup is of 32 bytes. If the check fails, we still continue executing the code which will lead to a panic as in the subsequent we try to index the bytes. The fix would be to ignore this tx when the validation fails fix: Fix tests which were failing (#16) This PR fixes all the failing tests in the astria-geth repo apart from the [`TestReimportMirroredState` ](https://github.com/astriaorg/astria-geth/blob/f5a95fe13d581ae1ec67caeaae6efe5338399a3e/consensus/clique/clique_test.go#L44) (More details at the end) The main reasons for the tests failing were: 1. #5 which transfers the basefee to the fee recipient rather than burning it. This caused all the tests which expect a certain state root to fail since they assume that the basefee would be burnt. It also failed tests which check for the balance of the fee recipient which fails because we also have to account for the base fee. To fix them, we replaced the tests with the expected state root and updated the tests with the expected balance of the fee recipient. 2. Picking txs from the `AstriaOrdered` rather than from the geth mempool. Lot of tests place the txs in the Geth mempool and they end up failing as we do not pick txx from their. To fix this, we had to add these txs to the `AstriaOrdered` mempool instead This PR also adds a github action to run the tests. `TestReimportMirroredState` is very tricky to fix since the fee recipient while building the block and while verifying the block before inserting onchain is different. This is because while verifying the block, we pick the fee recipient from the consensus engine's author which in this case is the signer of the block. It's tricker to fix it and we have commented it out for now. chore: migrate to depot to fix builds (#23) buildjet has not been working on this repo, and their customer support has been non responsive. feat: Add tests for execution grpc server and astria configs (#17) This PR adds the following: 1. Tests for astria execution grpc server 2. Tests for astria config 3. Abstraction of astria validation logic and unit tests for them. feat!: store celestia height updates (#22) Updates geth to work with proto changes here: https://github.com/astriaorg/astria/pull/1121/files Adds storage of the celestia height into the DB, so we don't resync celestia data already read on restart. ci: update the test ci to run build/ci.go (#27) Previously, the test ci was running `go test -short ./...`. Instead run the build/ci.go to better test on the ci implement `AstriaMintableERC20`, update deposit tx to support ERC20 mints (#20) - implement `AstriaMintableERC20.sol`, allows an immutable address `bridge` to mint or burn - this address should be set at genesis and the sender of `DepositTx`s is set to this address - `DepositTx` was updated to include `to` and `data` which are set for erc20 mints (later on burns), `to` is set to the erc20 address of the `AstriaMintableERC20` and `data` is the calldata to call the mint function - genesis config fields for erc20s must be set to allow bridging erc20s from the sequencer. namely the erc20 contract address must be set, meaning the token must already be deployed testing: - i tested this manually end-to-end using the instructions in `contracts/README.md`. closes #9
SuperFluffy
pushed a commit
that referenced
this pull request
Apr 28, 2025
Squash 916d6a4 to 3d160b2 3d160b2 is the last commit before the infamous squash + merge of geth into astria-geth The commit messages below are left unchanged for now so that they can be cleaned up later. Feature/grpc execution api (#1) * add buf config and generated grpc code * poc e2e grpc communication * comment out panicking code * logging help * now reads cli args. now stops grpc server on shutdown. * add mutex to GRPCServerHandler update readme containerize w/ github action to build and push to ghcr (#3) fix gh action syntax (#4) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax build and push manually (#5) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax * dont use github action, do it ourselves * push manually Feature/containerize (#6) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax * dont use github action, do it ourselves * push manually * correct multi line update SubmitTransaction to send tx to metro ethclient: ensure returned subscription is nil on error (#26976) core/state, trie: remove Try prefix in Trie accessors (#26975) This change renames StateTrie methods to remove the Try* prefix. We added the Trie methods with prefix 'Try' a long time ago, working around the problem that most existing methods of Trie did not return the database error. This weird naming convention has persisted until now. Co-authored-by: Gary Rong <garyrong0905@gmail.com> metrics/librato: ensure resp.body closed (#26969) This change ensures that we call Close on a http response body, in various places in the source code (mostly tests) core/vm: use atomic.Bool (#26951) Make use of new atomic types --------- Co-authored-by: Felix Lange <fjl@twurst.com> Co-authored-by: Martin Holst Swende <martin@swende.se> core/bloombits: use atomic type (#26993) core/state: use atomic.Bool (#26992) graphql: fix data races (#26965) Fixes multiple data races caused by the fact that resolving fields are done concurrently by the graphql library. It also enforces caching at the stateobject level for account fields. eth/tracers/native: prevent panic for LOG edge-cases (#26848) This PR fixes OOM panic in the callTracer as well as panicing on opcode validation errors (e.g. stack underflow) in callTracer and prestateTracer. Co-authored-by: Martin Holst Swende <martin@swende.se> internal/debug: add log.logfmt flag to set logging to use logfmt (#26970) docs: update outdated DeriveSha docs comment (#26968) add localnet genesis update metro-transactions dep Adding features to mempool to support our pre-ordered txs (#2) * Adding features to mempool to support our pre-ordered txs * Add framing for gRPC to execute blocks * Remove public engine API call * Fix circular dependencies * Updated to use new Init, and fill out starting attributes * Add clear astriaordered, cleanup * readme fix * add bash and jq to final docker image * make txpool interface * no more panics, update DoBlock to also update state + store block * cleanup * set post-merge at genesis * cleanup * doc update * remove txpool interface changes * cleanup * cleanup * build and push images wih tags defined by git tags * build for multiple architectures. use docker-metadata action for semver * add push: true * only build arm for git tags/releases --------- Co-authored-by: Jesse Snyder <jessetsnyder@gmail.com> Co-authored-by: elizabeth <elizabethjbinks@gmail.com> tag image with latest for builds from astria branch (#8) rename state_root to block_hash use FROM --platform=$BUILDPLATFORM to build arm images correctly (#9) use v4 of build and push action (#10) * use FROM --platform=$BUILDPLATFORM to build arm images correctly * use v4 of docker build and push fix arm builds. only build arm for tags and merges to default branch. (#12) * fix arm builds. only build arm for tags and merges to default branch. * fix gh action function syntax * more correct comment * build for semver tags Features and fixes needed for contract deployment test (#13) * add and remove tx from geth mempool so forge can deploy a contract * pass metro addr and port by flag * fixes from pr feedback * formatting implement FinalizeBlock gRPC call bump metro-transactions dep Update Protos (#16) * uses new protos * remove old protos update deps update deps no more unknown/unknown image (#19) * no more unknown/unknown image * hardcode condition Changing from prev_state_root to prev_block_hash remove metro integrate submission to sequencer log updates use env vars for chain id/tendermint endpoint rename to cometbft Make the environment variables actually work cleaner simple logging change removed sleep before getPayload by removing goroutine in buildPayload removed unneeded comment removed for loop from buildPayload Remove direct sequencer submission cleanup go.mod Migrate to the v1alpha1 API Missed pieces set genesis block as head/safe/final remove all mentions of metro Initial implementation of Execution v1alpha2 api Integrate updates remove need for consensus api Updates small updates Update grpc/execution/server.go Co-authored-by: noot <36753753+noot@users.noreply.github.com> logging remove unused dependency Mark the chain 'finalized' on startup updates txpool fixes, remove v1alpha1, use forkchoice some cleanup formatting and such cleanup think it's good add new fields to base example genesis Update gh action for release branch Add timestamp to block generated from header Add missing parent hash chore: add some metrics specifically around astria tx execution small refactor fix feat: everything needed for genesis fix review + metrics update to prod pushed buf fix after grpc update switch to new execution apis package feat: support new API utilizing rollup data update to latest proto update docker to login implement DepositTx type and handle in state transition and execute_block cleanup fixes to get deposits working, update genesis.json cleanup cleanup exit node if genesis astria fields are unset add bridge address and asset IDs to genesis and check during execution don't allow deposit or blob txs in sequenced data minimum changes required for fork (#2) Updates the codeowners and sets a new package repo for `astria-geth` repo. Once this is merged, I can push the astria specific tags and we will get built images for old tags on this repo. feat: update protos, config for fees, updated bridging (#1) Allow setting eip 1559 fee params (#3) Adds a per block height config to set some EIP1559 fee params (elasticity and denominator) + adds a min base fee parameter. collect the base fee instead of burning it (#5) EIP-1559 burns the base fee. This doesn't make sense for rollups. The set fee recipient should collect it. Chore: Add forkdiff gh-page (#14) we need a static webpage to show changes made to astria-geth fork, this should be used both internally and externally. using a [forkdiff](https://github.com/protolambda/forkdiff) tool to create a static webpage. add a fork.yaml file structs the changes view and define the commits to compare with pages.yaml workflow deploys to gh-pages Currently deploying on PR for testing. feat: remove txs which do not end up in a block out of the mempool (#15) When the txs we receive from `ExecuteBlocks` call fail to end up in a block, we keep them lying around the geth mempool but clear it from the astria mempool. This PR attempts to remove such txs from the geth mempool. chore: make server logs less noisy (#12) Moves the initial request logs to debug logs, reduces the amount of information printed, and ensures okay formatting. Logs from Geth were very messy, block hashes were messy bytes, and all the information from each request was logged out unnecessarily. fix: continue the tx validation loop if assetId length check fails (#18) Currently, we check if the length of the assetId sent by the rollup is of 32 bytes. If the check fails, we still continue executing the code which will lead to a panic as in the subsequent we try to index the bytes. The fix would be to ignore this tx when the validation fails fix: Fix tests which were failing (#16) This PR fixes all the failing tests in the astria-geth repo apart from the [`TestReimportMirroredState` ](https://github.com/astriaorg/astria-geth/blob/f5a95fe13d581ae1ec67caeaae6efe5338399a3e/consensus/clique/clique_test.go#L44) (More details at the end) The main reasons for the tests failing were: 1. #5 which transfers the basefee to the fee recipient rather than burning it. This caused all the tests which expect a certain state root to fail since they assume that the basefee would be burnt. It also failed tests which check for the balance of the fee recipient which fails because we also have to account for the base fee. To fix them, we replaced the tests with the expected state root and updated the tests with the expected balance of the fee recipient. 2. Picking txs from the `AstriaOrdered` rather than from the geth mempool. Lot of tests place the txs in the Geth mempool and they end up failing as we do not pick txx from their. To fix this, we had to add these txs to the `AstriaOrdered` mempool instead This PR also adds a github action to run the tests. `TestReimportMirroredState` is very tricky to fix since the fee recipient while building the block and while verifying the block before inserting onchain is different. This is because while verifying the block, we pick the fee recipient from the consensus engine's author which in this case is the signer of the block. It's tricker to fix it and we have commented it out for now. chore: migrate to depot to fix builds (#23) buildjet has not been working on this repo, and their customer support has been non responsive. feat: Add tests for execution grpc server and astria configs (#17) This PR adds the following: 1. Tests for astria execution grpc server 2. Tests for astria config 3. Abstraction of astria validation logic and unit tests for them. feat!: store celestia height updates (#22) Updates geth to work with proto changes here: https://github.com/astriaorg/astria/pull/1121/files Adds storage of the celestia height into the DB, so we don't resync celestia data already read on restart. ci: update the test ci to run build/ci.go (#27) Previously, the test ci was running `go test -short ./...`. Instead run the build/ci.go to better test on the ci implement `AstriaMintableERC20`, update deposit tx to support ERC20 mints (#20) - implement `AstriaMintableERC20.sol`, allows an immutable address `bridge` to mint or burn - this address should be set at genesis and the sender of `DepositTx`s is set to this address - `DepositTx` was updated to include `to` and `data` which are set for erc20 mints (later on burns), `to` is set to the erc20 address of the `AstriaMintableERC20` and `data` is the calldata to call the mint function - genesis config fields for erc20s must be set to allow bridging erc20s from the sequencer. namely the erc20 contract address must be set, meaning the token must already be deployed testing: - i tested this manually end-to-end using the instructions in `contracts/README.md`. closes #9
SuperFluffy
pushed a commit
that referenced
this pull request
Apr 28, 2025
3d160b2 is the last commit before the infamous squash + merge of geth into astria-geth The commit messages below are left unchanged for now so that they can be cleaned up later. Feature/grpc execution api (#1) * add buf config and generated grpc code * poc e2e grpc communication * comment out panicking code * logging help * now reads cli args. now stops grpc server on shutdown. * add mutex to GRPCServerHandler update readme containerize w/ github action to build and push to ghcr (#3) fix gh action syntax (#4) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax build and push manually (#5) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax * dont use github action, do it ourselves * push manually Feature/containerize (#6) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax * dont use github action, do it ourselves * push manually * correct multi line update SubmitTransaction to send tx to metro ethclient: ensure returned subscription is nil on error (#26976) core/state, trie: remove Try prefix in Trie accessors (#26975) This change renames StateTrie methods to remove the Try* prefix. We added the Trie methods with prefix 'Try' a long time ago, working around the problem that most existing methods of Trie did not return the database error. This weird naming convention has persisted until now. Co-authored-by: Gary Rong <garyrong0905@gmail.com> metrics/librato: ensure resp.body closed (#26969) This change ensures that we call Close on a http response body, in various places in the source code (mostly tests) core/vm: use atomic.Bool (#26951) Make use of new atomic types --------- Co-authored-by: Felix Lange <fjl@twurst.com> Co-authored-by: Martin Holst Swende <martin@swende.se> core/bloombits: use atomic type (#26993) core/state: use atomic.Bool (#26992) graphql: fix data races (#26965) Fixes multiple data races caused by the fact that resolving fields are done concurrently by the graphql library. It also enforces caching at the stateobject level for account fields. eth/tracers/native: prevent panic for LOG edge-cases (#26848) This PR fixes OOM panic in the callTracer as well as panicing on opcode validation errors (e.g. stack underflow) in callTracer and prestateTracer. Co-authored-by: Martin Holst Swende <martin@swende.se> internal/debug: add log.logfmt flag to set logging to use logfmt (#26970) docs: update outdated DeriveSha docs comment (#26968) add localnet genesis update metro-transactions dep Adding features to mempool to support our pre-ordered txs (#2) * Adding features to mempool to support our pre-ordered txs * Add framing for gRPC to execute blocks * Remove public engine API call * Fix circular dependencies * Updated to use new Init, and fill out starting attributes * Add clear astriaordered, cleanup * readme fix * add bash and jq to final docker image * make txpool interface * no more panics, update DoBlock to also update state + store block * cleanup * set post-merge at genesis * cleanup * doc update * remove txpool interface changes * cleanup * cleanup * build and push images wih tags defined by git tags * build for multiple architectures. use docker-metadata action for semver * add push: true * only build arm for git tags/releases --------- Co-authored-by: Jesse Snyder <jessetsnyder@gmail.com> Co-authored-by: elizabeth <elizabethjbinks@gmail.com> tag image with latest for builds from astria branch (#8) rename state_root to block_hash use FROM --platform=$BUILDPLATFORM to build arm images correctly (#9) use v4 of build and push action (#10) * use FROM --platform=$BUILDPLATFORM to build arm images correctly * use v4 of docker build and push fix arm builds. only build arm for tags and merges to default branch. (#12) * fix arm builds. only build arm for tags and merges to default branch. * fix gh action function syntax * more correct comment * build for semver tags Features and fixes needed for contract deployment test (#13) * add and remove tx from geth mempool so forge can deploy a contract * pass metro addr and port by flag * fixes from pr feedback * formatting implement FinalizeBlock gRPC call bump metro-transactions dep Update Protos (#16) * uses new protos * remove old protos update deps update deps no more unknown/unknown image (#19) * no more unknown/unknown image * hardcode condition Changing from prev_state_root to prev_block_hash remove metro integrate submission to sequencer log updates use env vars for chain id/tendermint endpoint rename to cometbft Make the environment variables actually work cleaner simple logging change removed sleep before getPayload by removing goroutine in buildPayload removed unneeded comment removed for loop from buildPayload Remove direct sequencer submission cleanup go.mod Migrate to the v1alpha1 API Missed pieces set genesis block as head/safe/final remove all mentions of metro Initial implementation of Execution v1alpha2 api Integrate updates remove need for consensus api Updates small updates Update grpc/execution/server.go Co-authored-by: noot <36753753+noot@users.noreply.github.com> logging remove unused dependency Mark the chain 'finalized' on startup updates txpool fixes, remove v1alpha1, use forkchoice some cleanup formatting and such cleanup think it's good add new fields to base example genesis Update gh action for release branch Add timestamp to block generated from header Add missing parent hash chore: add some metrics specifically around astria tx execution small refactor fix feat: everything needed for genesis fix review + metrics update to prod pushed buf fix after grpc update switch to new execution apis package feat: support new API utilizing rollup data update to latest proto update docker to login implement DepositTx type and handle in state transition and execute_block cleanup fixes to get deposits working, update genesis.json cleanup cleanup exit node if genesis astria fields are unset add bridge address and asset IDs to genesis and check during execution don't allow deposit or blob txs in sequenced data minimum changes required for fork (#2) Updates the codeowners and sets a new package repo for `astria-geth` repo. Once this is merged, I can push the astria specific tags and we will get built images for old tags on this repo. feat: update protos, config for fees, updated bridging (#1) Allow setting eip 1559 fee params (#3) Adds a per block height config to set some EIP1559 fee params (elasticity and denominator) + adds a min base fee parameter. collect the base fee instead of burning it (#5) EIP-1559 burns the base fee. This doesn't make sense for rollups. The set fee recipient should collect it. Chore: Add forkdiff gh-page (#14) we need a static webpage to show changes made to astria-geth fork, this should be used both internally and externally. using a [forkdiff](https://github.com/protolambda/forkdiff) tool to create a static webpage. add a fork.yaml file structs the changes view and define the commits to compare with pages.yaml workflow deploys to gh-pages Currently deploying on PR for testing. feat: remove txs which do not end up in a block out of the mempool (#15) When the txs we receive from `ExecuteBlocks` call fail to end up in a block, we keep them lying around the geth mempool but clear it from the astria mempool. This PR attempts to remove such txs from the geth mempool. chore: make server logs less noisy (#12) Moves the initial request logs to debug logs, reduces the amount of information printed, and ensures okay formatting. Logs from Geth were very messy, block hashes were messy bytes, and all the information from each request was logged out unnecessarily. fix: continue the tx validation loop if assetId length check fails (#18) Currently, we check if the length of the assetId sent by the rollup is of 32 bytes. If the check fails, we still continue executing the code which will lead to a panic as in the subsequent we try to index the bytes. The fix would be to ignore this tx when the validation fails fix: Fix tests which were failing (#16) This PR fixes all the failing tests in the astria-geth repo apart from the [`TestReimportMirroredState` ](https://github.com/astriaorg/astria-geth/blob/f5a95fe13d581ae1ec67caeaae6efe5338399a3e/consensus/clique/clique_test.go#L44) (More details at the end) The main reasons for the tests failing were: 1. #5 which transfers the basefee to the fee recipient rather than burning it. This caused all the tests which expect a certain state root to fail since they assume that the basefee would be burnt. It also failed tests which check for the balance of the fee recipient which fails because we also have to account for the base fee. To fix them, we replaced the tests with the expected state root and updated the tests with the expected balance of the fee recipient. 2. Picking txs from the `AstriaOrdered` rather than from the geth mempool. Lot of tests place the txs in the Geth mempool and they end up failing as we do not pick txx from their. To fix this, we had to add these txs to the `AstriaOrdered` mempool instead This PR also adds a github action to run the tests. `TestReimportMirroredState` is very tricky to fix since the fee recipient while building the block and while verifying the block before inserting onchain is different. This is because while verifying the block, we pick the fee recipient from the consensus engine's author which in this case is the signer of the block. It's tricker to fix it and we have commented it out for now. chore: migrate to depot to fix builds (#23) buildjet has not been working on this repo, and their customer support has been non responsive. feat: Add tests for execution grpc server and astria configs (#17) This PR adds the following: 1. Tests for astria execution grpc server 2. Tests for astria config 3. Abstraction of astria validation logic and unit tests for them. feat!: store celestia height updates (#22) Updates geth to work with proto changes here: https://github.com/astriaorg/astria/pull/1121/files Adds storage of the celestia height into the DB, so we don't resync celestia data already read on restart. ci: update the test ci to run build/ci.go (#27) Previously, the test ci was running `go test -short ./...`. Instead run the build/ci.go to better test on the ci implement `AstriaMintableERC20`, update deposit tx to support ERC20 mints (#20) - implement `AstriaMintableERC20.sol`, allows an immutable address `bridge` to mint or burn - this address should be set at genesis and the sender of `DepositTx`s is set to this address - `DepositTx` was updated to include `to` and `data` which are set for erc20 mints (later on burns), `to` is set to the erc20 address of the `AstriaMintableERC20` and `data` is the calldata to call the mint function - genesis config fields for erc20s must be set to allow bridging erc20s from the sequencer. namely the erc20 contract address must be set, meaning the token must already be deployed testing: - i tested this manually end-to-end using the instructions in `contracts/README.md`. closes #9
SuperFluffy
pushed a commit
that referenced
this pull request
Apr 28, 2025
rebase on top of geth v1.13.6 Squash 916d6a4 to 3d160b2 3d160b2 is the last commit before the infamous squash + merge of geth into astria-geth The commit messages below are left unchanged for now so that they can be cleaned up later. Feature/grpc execution api (#1) * add buf config and generated grpc code * poc e2e grpc communication * comment out panicking code * logging help * now reads cli args. now stops grpc server on shutdown. * add mutex to GRPCServerHandler update readme containerize w/ github action to build and push to ghcr (#3) fix gh action syntax (#4) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax build and push manually (#5) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax * dont use github action, do it ourselves * push manually Feature/containerize (#6) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax * dont use github action, do it ourselves * push manually * correct multi line update SubmitTransaction to send tx to metro ethclient: ensure returned subscription is nil on error (#26976) core/state, trie: remove Try prefix in Trie accessors (#26975) This change renames StateTrie methods to remove the Try* prefix. We added the Trie methods with prefix 'Try' a long time ago, working around the problem that most existing methods of Trie did not return the database error. This weird naming convention has persisted until now. Co-authored-by: Gary Rong <garyrong0905@gmail.com> metrics/librato: ensure resp.body closed (#26969) This change ensures that we call Close on a http response body, in various places in the source code (mostly tests) core/vm: use atomic.Bool (#26951) Make use of new atomic types --------- Co-authored-by: Felix Lange <fjl@twurst.com> Co-authored-by: Martin Holst Swende <martin@swende.se> core/bloombits: use atomic type (#26993) core/state: use atomic.Bool (#26992) graphql: fix data races (#26965) Fixes multiple data races caused by the fact that resolving fields are done concurrently by the graphql library. It also enforces caching at the stateobject level for account fields. eth/tracers/native: prevent panic for LOG edge-cases (#26848) This PR fixes OOM panic in the callTracer as well as panicing on opcode validation errors (e.g. stack underflow) in callTracer and prestateTracer. Co-authored-by: Martin Holst Swende <martin@swende.se> internal/debug: add log.logfmt flag to set logging to use logfmt (#26970) docs: update outdated DeriveSha docs comment (#26968) add localnet genesis update metro-transactions dep Adding features to mempool to support our pre-ordered txs (#2) * Adding features to mempool to support our pre-ordered txs * Add framing for gRPC to execute blocks * Remove public engine API call * Fix circular dependencies * Updated to use new Init, and fill out starting attributes * Add clear astriaordered, cleanup * readme fix * add bash and jq to final docker image * make txpool interface * no more panics, update DoBlock to also update state + store block * cleanup * set post-merge at genesis * cleanup * doc update * remove txpool interface changes * cleanup * cleanup * build and push images wih tags defined by git tags * build for multiple architectures. use docker-metadata action for semver * add push: true * only build arm for git tags/releases --------- Co-authored-by: Jesse Snyder <jessetsnyder@gmail.com> Co-authored-by: elizabeth <elizabethjbinks@gmail.com> tag image with latest for builds from astria branch (#8) rename state_root to block_hash use FROM --platform=$BUILDPLATFORM to build arm images correctly (#9) use v4 of build and push action (#10) * use FROM --platform=$BUILDPLATFORM to build arm images correctly * use v4 of docker build and push fix arm builds. only build arm for tags and merges to default branch. (#12) * fix arm builds. only build arm for tags and merges to default branch. * fix gh action function syntax * more correct comment * build for semver tags Features and fixes needed for contract deployment test (#13) * add and remove tx from geth mempool so forge can deploy a contract * pass metro addr and port by flag * fixes from pr feedback * formatting implement FinalizeBlock gRPC call bump metro-transactions dep Update Protos (#16) * uses new protos * remove old protos update deps update deps no more unknown/unknown image (#19) * no more unknown/unknown image * hardcode condition Changing from prev_state_root to prev_block_hash remove metro integrate submission to sequencer log updates use env vars for chain id/tendermint endpoint rename to cometbft Make the environment variables actually work cleaner simple logging change removed sleep before getPayload by removing goroutine in buildPayload removed unneeded comment removed for loop from buildPayload Remove direct sequencer submission cleanup go.mod Migrate to the v1alpha1 API Missed pieces set genesis block as head/safe/final remove all mentions of metro Initial implementation of Execution v1alpha2 api Integrate updates remove need for consensus api Updates small updates Update grpc/execution/server.go Co-authored-by: noot <36753753+noot@users.noreply.github.com> logging remove unused dependency Mark the chain 'finalized' on startup updates txpool fixes, remove v1alpha1, use forkchoice some cleanup formatting and such cleanup think it's good add new fields to base example genesis Update gh action for release branch Add timestamp to block generated from header Add missing parent hash chore: add some metrics specifically around astria tx execution small refactor fix feat: everything needed for genesis fix review + metrics update to prod pushed buf fix after grpc update switch to new execution apis package feat: support new API utilizing rollup data update to latest proto update docker to login implement DepositTx type and handle in state transition and execute_block cleanup fixes to get deposits working, update genesis.json cleanup cleanup exit node if genesis astria fields are unset add bridge address and asset IDs to genesis and check during execution don't allow deposit or blob txs in sequenced data minimum changes required for fork (#2) Updates the codeowners and sets a new package repo for `astria-geth` repo. Once this is merged, I can push the astria specific tags and we will get built images for old tags on this repo. feat: update protos, config for fees, updated bridging (#1) Allow setting eip 1559 fee params (#3) Adds a per block height config to set some EIP1559 fee params (elasticity and denominator) + adds a min base fee parameter. collect the base fee instead of burning it (#5) EIP-1559 burns the base fee. This doesn't make sense for rollups. The set fee recipient should collect it. Chore: Add forkdiff gh-page (#14) we need a static webpage to show changes made to astria-geth fork, this should be used both internally and externally. using a [forkdiff](https://github.com/protolambda/forkdiff) tool to create a static webpage. add a fork.yaml file structs the changes view and define the commits to compare with pages.yaml workflow deploys to gh-pages Currently deploying on PR for testing. feat: remove txs which do not end up in a block out of the mempool (#15) When the txs we receive from `ExecuteBlocks` call fail to end up in a block, we keep them lying around the geth mempool but clear it from the astria mempool. This PR attempts to remove such txs from the geth mempool. chore: make server logs less noisy (#12) Moves the initial request logs to debug logs, reduces the amount of information printed, and ensures okay formatting. Logs from Geth were very messy, block hashes were messy bytes, and all the information from each request was logged out unnecessarily. fix: continue the tx validation loop if assetId length check fails (#18) Currently, we check if the length of the assetId sent by the rollup is of 32 bytes. If the check fails, we still continue executing the code which will lead to a panic as in the subsequent we try to index the bytes. The fix would be to ignore this tx when the validation fails fix: Fix tests which were failing (#16) This PR fixes all the failing tests in the astria-geth repo apart from the [`TestReimportMirroredState` ](https://github.com/astriaorg/astria-geth/blob/f5a95fe13d581ae1ec67caeaae6efe5338399a3e/consensus/clique/clique_test.go#L44) (More details at the end) The main reasons for the tests failing were: 1. #5 which transfers the basefee to the fee recipient rather than burning it. This caused all the tests which expect a certain state root to fail since they assume that the basefee would be burnt. It also failed tests which check for the balance of the fee recipient which fails because we also have to account for the base fee. To fix them, we replaced the tests with the expected state root and updated the tests with the expected balance of the fee recipient. 2. Picking txs from the `AstriaOrdered` rather than from the geth mempool. Lot of tests place the txs in the Geth mempool and they end up failing as we do not pick txx from their. To fix this, we had to add these txs to the `AstriaOrdered` mempool instead This PR also adds a github action to run the tests. `TestReimportMirroredState` is very tricky to fix since the fee recipient while building the block and while verifying the block before inserting onchain is different. This is because while verifying the block, we pick the fee recipient from the consensus engine's author which in this case is the signer of the block. It's tricker to fix it and we have commented it out for now. chore: migrate to depot to fix builds (#23) buildjet has not been working on this repo, and their customer support has been non responsive. feat: Add tests for execution grpc server and astria configs (#17) This PR adds the following: 1. Tests for astria execution grpc server 2. Tests for astria config 3. Abstraction of astria validation logic and unit tests for them. feat!: store celestia height updates (#22) Updates geth to work with proto changes here: https://github.com/astriaorg/astria/pull/1121/files Adds storage of the celestia height into the DB, so we don't resync celestia data already read on restart. ci: update the test ci to run build/ci.go (#27) Previously, the test ci was running `go test -short ./...`. Instead run the build/ci.go to better test on the ci implement `AstriaMintableERC20`, update deposit tx to support ERC20 mints (#20) - implement `AstriaMintableERC20.sol`, allows an immutable address `bridge` to mint or burn - this address should be set at genesis and the sender of `DepositTx`s is set to this address - `DepositTx` was updated to include `to` and `data` which are set for erc20 mints (later on burns), `to` is set to the erc20 address of the `AstriaMintableERC20` and `data` is the calldata to call the mint function - genesis config fields for erc20s must be set to allow bridging erc20s from the sequencer. namely the erc20 contract address must be set, meaning the token must already be deployed testing: - i tested this manually end-to-end using the instructions in `contracts/README.md`. closes #9
SuperFluffy
pushed a commit
that referenced
this pull request
Apr 28, 2025
rebase on top of geth v1.13.6 Squash 916d6a4 to 3d160b2 3d160b2 is the last commit before the infamous squash + merge of geth into astria-geth The commit messages below are left unchanged for now so that they can be cleaned up later. Feature/grpc execution api (#1) * add buf config and generated grpc code * poc e2e grpc communication * comment out panicking code * logging help * now reads cli args. now stops grpc server on shutdown. * add mutex to GRPCServerHandler update readme containerize w/ github action to build and push to ghcr (#3) fix gh action syntax (#4) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax build and push manually (#5) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax * dont use github action, do it ourselves * push manually Feature/containerize (#6) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax * dont use github action, do it ourselves * push manually * correct multi line update SubmitTransaction to send tx to metro ethclient: ensure returned subscription is nil on error (#26976) core/state, trie: remove Try prefix in Trie accessors (#26975) This change renames StateTrie methods to remove the Try* prefix. We added the Trie methods with prefix 'Try' a long time ago, working around the problem that most existing methods of Trie did not return the database error. This weird naming convention has persisted until now. Co-authored-by: Gary Rong <garyrong0905@gmail.com> metrics/librato: ensure resp.body closed (#26969) This change ensures that we call Close on a http response body, in various places in the source code (mostly tests) core/vm: use atomic.Bool (#26951) Make use of new atomic types --------- Co-authored-by: Felix Lange <fjl@twurst.com> Co-authored-by: Martin Holst Swende <martin@swende.se> core/bloombits: use atomic type (#26993) core/state: use atomic.Bool (#26992) graphql: fix data races (#26965) Fixes multiple data races caused by the fact that resolving fields are done concurrently by the graphql library. It also enforces caching at the stateobject level for account fields. eth/tracers/native: prevent panic for LOG edge-cases (#26848) This PR fixes OOM panic in the callTracer as well as panicing on opcode validation errors (e.g. stack underflow) in callTracer and prestateTracer. Co-authored-by: Martin Holst Swende <martin@swende.se> internal/debug: add log.logfmt flag to set logging to use logfmt (#26970) docs: update outdated DeriveSha docs comment (#26968) add localnet genesis update metro-transactions dep Adding features to mempool to support our pre-ordered txs (#2) * Adding features to mempool to support our pre-ordered txs * Add framing for gRPC to execute blocks * Remove public engine API call * Fix circular dependencies * Updated to use new Init, and fill out starting attributes * Add clear astriaordered, cleanup * readme fix * add bash and jq to final docker image * make txpool interface * no more panics, update DoBlock to also update state + store block * cleanup * set post-merge at genesis * cleanup * doc update * remove txpool interface changes * cleanup * cleanup * build and push images wih tags defined by git tags * build for multiple architectures. use docker-metadata action for semver * add push: true * only build arm for git tags/releases --------- Co-authored-by: Jesse Snyder <jessetsnyder@gmail.com> Co-authored-by: elizabeth <elizabethjbinks@gmail.com> tag image with latest for builds from astria branch (#8) rename state_root to block_hash use FROM --platform=$BUILDPLATFORM to build arm images correctly (#9) use v4 of build and push action (#10) * use FROM --platform=$BUILDPLATFORM to build arm images correctly * use v4 of docker build and push fix arm builds. only build arm for tags and merges to default branch. (#12) * fix arm builds. only build arm for tags and merges to default branch. * fix gh action function syntax * more correct comment * build for semver tags Features and fixes needed for contract deployment test (#13) * add and remove tx from geth mempool so forge can deploy a contract * pass metro addr and port by flag * fixes from pr feedback * formatting implement FinalizeBlock gRPC call bump metro-transactions dep Update Protos (#16) * uses new protos * remove old protos update deps update deps no more unknown/unknown image (#19) * no more unknown/unknown image * hardcode condition Changing from prev_state_root to prev_block_hash remove metro integrate submission to sequencer log updates use env vars for chain id/tendermint endpoint rename to cometbft Make the environment variables actually work cleaner simple logging change removed sleep before getPayload by removing goroutine in buildPayload removed unneeded comment removed for loop from buildPayload Remove direct sequencer submission cleanup go.mod Migrate to the v1alpha1 API Missed pieces set genesis block as head/safe/final remove all mentions of metro Initial implementation of Execution v1alpha2 api Integrate updates remove need for consensus api Updates small updates Update grpc/execution/server.go Co-authored-by: noot <36753753+noot@users.noreply.github.com> logging remove unused dependency Mark the chain 'finalized' on startup updates txpool fixes, remove v1alpha1, use forkchoice some cleanup formatting and such cleanup think it's good add new fields to base example genesis Update gh action for release branch Add timestamp to block generated from header Add missing parent hash chore: add some metrics specifically around astria tx execution small refactor fix feat: everything needed for genesis fix review + metrics update to prod pushed buf fix after grpc update switch to new execution apis package feat: support new API utilizing rollup data update to latest proto update docker to login implement DepositTx type and handle in state transition and execute_block cleanup fixes to get deposits working, update genesis.json cleanup cleanup exit node if genesis astria fields are unset add bridge address and asset IDs to genesis and check during execution don't allow deposit or blob txs in sequenced data minimum changes required for fork (#2) Updates the codeowners and sets a new package repo for `astria-geth` repo. Once this is merged, I can push the astria specific tags and we will get built images for old tags on this repo. feat: update protos, config for fees, updated bridging (#1) Allow setting eip 1559 fee params (#3) Adds a per block height config to set some EIP1559 fee params (elasticity and denominator) + adds a min base fee parameter. collect the base fee instead of burning it (#5) EIP-1559 burns the base fee. This doesn't make sense for rollups. The set fee recipient should collect it. Chore: Add forkdiff gh-page (#14) we need a static webpage to show changes made to astria-geth fork, this should be used both internally and externally. using a [forkdiff](https://github.com/protolambda/forkdiff) tool to create a static webpage. add a fork.yaml file structs the changes view and define the commits to compare with pages.yaml workflow deploys to gh-pages Currently deploying on PR for testing. feat: remove txs which do not end up in a block out of the mempool (#15) When the txs we receive from `ExecuteBlocks` call fail to end up in a block, we keep them lying around the geth mempool but clear it from the astria mempool. This PR attempts to remove such txs from the geth mempool. chore: make server logs less noisy (#12) Moves the initial request logs to debug logs, reduces the amount of information printed, and ensures okay formatting. Logs from Geth were very messy, block hashes were messy bytes, and all the information from each request was logged out unnecessarily. fix: continue the tx validation loop if assetId length check fails (#18) Currently, we check if the length of the assetId sent by the rollup is of 32 bytes. If the check fails, we still continue executing the code which will lead to a panic as in the subsequent we try to index the bytes. The fix would be to ignore this tx when the validation fails fix: Fix tests which were failing (#16) This PR fixes all the failing tests in the astria-geth repo apart from the [`TestReimportMirroredState` ](https://github.com/astriaorg/astria-geth/blob/f5a95fe13d581ae1ec67caeaae6efe5338399a3e/consensus/clique/clique_test.go#L44) (More details at the end) The main reasons for the tests failing were: 1. #5 which transfers the basefee to the fee recipient rather than burning it. This caused all the tests which expect a certain state root to fail since they assume that the basefee would be burnt. It also failed tests which check for the balance of the fee recipient which fails because we also have to account for the base fee. To fix them, we replaced the tests with the expected state root and updated the tests with the expected balance of the fee recipient. 2. Picking txs from the `AstriaOrdered` rather than from the geth mempool. Lot of tests place the txs in the Geth mempool and they end up failing as we do not pick txx from their. To fix this, we had to add these txs to the `AstriaOrdered` mempool instead This PR also adds a github action to run the tests. `TestReimportMirroredState` is very tricky to fix since the fee recipient while building the block and while verifying the block before inserting onchain is different. This is because while verifying the block, we pick the fee recipient from the consensus engine's author which in this case is the signer of the block. It's tricker to fix it and we have commented it out for now. chore: migrate to depot to fix builds (#23) buildjet has not been working on this repo, and their customer support has been non responsive. feat: Add tests for execution grpc server and astria configs (#17) This PR adds the following: 1. Tests for astria execution grpc server 2. Tests for astria config 3. Abstraction of astria validation logic and unit tests for them. feat!: store celestia height updates (#22) Updates geth to work with proto changes here: https://github.com/astriaorg/astria/pull/1121/files Adds storage of the celestia height into the DB, so we don't resync celestia data already read on restart. ci: update the test ci to run build/ci.go (#27) Previously, the test ci was running `go test -short ./...`. Instead run the build/ci.go to better test on the ci implement `AstriaMintableERC20`, update deposit tx to support ERC20 mints (#20) - implement `AstriaMintableERC20.sol`, allows an immutable address `bridge` to mint or burn - this address should be set at genesis and the sender of `DepositTx`s is set to this address - `DepositTx` was updated to include `to` and `data` which are set for erc20 mints (later on burns), `to` is set to the erc20 address of the `AstriaMintableERC20` and `data` is the calldata to call the mint function - genesis config fields for erc20s must be set to allow bridging erc20s from the sequencer. namely the erc20 contract address must be set, meaning the token must already be deployed testing: - i tested this manually end-to-end using the instructions in `contracts/README.md`. closes #9
SuperFluffy
pushed a commit
that referenced
this pull request
Apr 28, 2025
rebase on top of geth v1.13.6 Squash 916d6a4 to 3d160b2 3d160b2 is the last commit before the infamous squash + merge of geth into astria-geth The commit messages below are left unchanged for now so that they can be cleaned up later. Feature/grpc execution api (#1) * add buf config and generated grpc code * poc e2e grpc communication * comment out panicking code * logging help * now reads cli args. now stops grpc server on shutdown. * add mutex to GRPCServerHandler update readme containerize w/ github action to build and push to ghcr (#3) fix gh action syntax (#4) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax build and push manually (#5) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax * dont use github action, do it ourselves * push manually Feature/containerize (#6) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax * dont use github action, do it ourselves * push manually * correct multi line update SubmitTransaction to send tx to metro ethclient: ensure returned subscription is nil on error (#26976) core/state, trie: remove Try prefix in Trie accessors (#26975) This change renames StateTrie methods to remove the Try* prefix. We added the Trie methods with prefix 'Try' a long time ago, working around the problem that most existing methods of Trie did not return the database error. This weird naming convention has persisted until now. Co-authored-by: Gary Rong <garyrong0905@gmail.com> metrics/librato: ensure resp.body closed (#26969) This change ensures that we call Close on a http response body, in various places in the source code (mostly tests) core/vm: use atomic.Bool (#26951) Make use of new atomic types --------- Co-authored-by: Felix Lange <fjl@twurst.com> Co-authored-by: Martin Holst Swende <martin@swende.se> core/bloombits: use atomic type (#26993) core/state: use atomic.Bool (#26992) graphql: fix data races (#26965) Fixes multiple data races caused by the fact that resolving fields are done concurrently by the graphql library. It also enforces caching at the stateobject level for account fields. eth/tracers/native: prevent panic for LOG edge-cases (#26848) This PR fixes OOM panic in the callTracer as well as panicing on opcode validation errors (e.g. stack underflow) in callTracer and prestateTracer. Co-authored-by: Martin Holst Swende <martin@swende.se> internal/debug: add log.logfmt flag to set logging to use logfmt (#26970) docs: update outdated DeriveSha docs comment (#26968) add localnet genesis update metro-transactions dep Adding features to mempool to support our pre-ordered txs (#2) * Adding features to mempool to support our pre-ordered txs * Add framing for gRPC to execute blocks * Remove public engine API call * Fix circular dependencies * Updated to use new Init, and fill out starting attributes * Add clear astriaordered, cleanup * readme fix * add bash and jq to final docker image * make txpool interface * no more panics, update DoBlock to also update state + store block * cleanup * set post-merge at genesis * cleanup * doc update * remove txpool interface changes * cleanup * cleanup * build and push images wih tags defined by git tags * build for multiple architectures. use docker-metadata action for semver * add push: true * only build arm for git tags/releases --------- Co-authored-by: Jesse Snyder <jessetsnyder@gmail.com> Co-authored-by: elizabeth <elizabethjbinks@gmail.com> tag image with latest for builds from astria branch (#8) rename state_root to block_hash use FROM --platform=$BUILDPLATFORM to build arm images correctly (#9) use v4 of build and push action (#10) * use FROM --platform=$BUILDPLATFORM to build arm images correctly * use v4 of docker build and push fix arm builds. only build arm for tags and merges to default branch. (#12) * fix arm builds. only build arm for tags and merges to default branch. * fix gh action function syntax * more correct comment * build for semver tags Features and fixes needed for contract deployment test (#13) * add and remove tx from geth mempool so forge can deploy a contract * pass metro addr and port by flag * fixes from pr feedback * formatting implement FinalizeBlock gRPC call bump metro-transactions dep Update Protos (#16) * uses new protos * remove old protos update deps update deps no more unknown/unknown image (#19) * no more unknown/unknown image * hardcode condition Changing from prev_state_root to prev_block_hash remove metro integrate submission to sequencer log updates use env vars for chain id/tendermint endpoint rename to cometbft Make the environment variables actually work cleaner simple logging change removed sleep before getPayload by removing goroutine in buildPayload removed unneeded comment removed for loop from buildPayload Remove direct sequencer submission cleanup go.mod Migrate to the v1alpha1 API Missed pieces set genesis block as head/safe/final remove all mentions of metro Initial implementation of Execution v1alpha2 api Integrate updates remove need for consensus api Updates small updates Update grpc/execution/server.go Co-authored-by: noot <36753753+noot@users.noreply.github.com> logging remove unused dependency Mark the chain 'finalized' on startup updates txpool fixes, remove v1alpha1, use forkchoice some cleanup formatting and such cleanup think it's good add new fields to base example genesis Update gh action for release branch Add timestamp to block generated from header Add missing parent hash chore: add some metrics specifically around astria tx execution small refactor fix feat: everything needed for genesis fix review + metrics update to prod pushed buf fix after grpc update switch to new execution apis package feat: support new API utilizing rollup data update to latest proto update docker to login implement DepositTx type and handle in state transition and execute_block cleanup fixes to get deposits working, update genesis.json cleanup cleanup exit node if genesis astria fields are unset add bridge address and asset IDs to genesis and check during execution don't allow deposit or blob txs in sequenced data minimum changes required for fork (#2) Updates the codeowners and sets a new package repo for `astria-geth` repo. Once this is merged, I can push the astria specific tags and we will get built images for old tags on this repo. feat: update protos, config for fees, updated bridging (#1) Allow setting eip 1559 fee params (#3) Adds a per block height config to set some EIP1559 fee params (elasticity and denominator) + adds a min base fee parameter. collect the base fee instead of burning it (#5) EIP-1559 burns the base fee. This doesn't make sense for rollups. The set fee recipient should collect it. Chore: Add forkdiff gh-page (#14) we need a static webpage to show changes made to astria-geth fork, this should be used both internally and externally. using a [forkdiff](https://github.com/protolambda/forkdiff) tool to create a static webpage. add a fork.yaml file structs the changes view and define the commits to compare with pages.yaml workflow deploys to gh-pages Currently deploying on PR for testing. feat: remove txs which do not end up in a block out of the mempool (#15) When the txs we receive from `ExecuteBlocks` call fail to end up in a block, we keep them lying around the geth mempool but clear it from the astria mempool. This PR attempts to remove such txs from the geth mempool. chore: make server logs less noisy (#12) Moves the initial request logs to debug logs, reduces the amount of information printed, and ensures okay formatting. Logs from Geth were very messy, block hashes were messy bytes, and all the information from each request was logged out unnecessarily. fix: continue the tx validation loop if assetId length check fails (#18) Currently, we check if the length of the assetId sent by the rollup is of 32 bytes. If the check fails, we still continue executing the code which will lead to a panic as in the subsequent we try to index the bytes. The fix would be to ignore this tx when the validation fails fix: Fix tests which were failing (#16) This PR fixes all the failing tests in the astria-geth repo apart from the [`TestReimportMirroredState` ](https://github.com/astriaorg/astria-geth/blob/f5a95fe13d581ae1ec67caeaae6efe5338399a3e/consensus/clique/clique_test.go#L44) (More details at the end) The main reasons for the tests failing were: 1. #5 which transfers the basefee to the fee recipient rather than burning it. This caused all the tests which expect a certain state root to fail since they assume that the basefee would be burnt. It also failed tests which check for the balance of the fee recipient which fails because we also have to account for the base fee. To fix them, we replaced the tests with the expected state root and updated the tests with the expected balance of the fee recipient. 2. Picking txs from the `AstriaOrdered` rather than from the geth mempool. Lot of tests place the txs in the Geth mempool and they end up failing as we do not pick txx from their. To fix this, we had to add these txs to the `AstriaOrdered` mempool instead This PR also adds a github action to run the tests. `TestReimportMirroredState` is very tricky to fix since the fee recipient while building the block and while verifying the block before inserting onchain is different. This is because while verifying the block, we pick the fee recipient from the consensus engine's author which in this case is the signer of the block. It's tricker to fix it and we have commented it out for now. chore: migrate to depot to fix builds (#23) buildjet has not been working on this repo, and their customer support has been non responsive. feat: Add tests for execution grpc server and astria configs (#17) This PR adds the following: 1. Tests for astria execution grpc server 2. Tests for astria config 3. Abstraction of astria validation logic and unit tests for them. feat!: store celestia height updates (#22) Updates geth to work with proto changes here: https://github.com/astriaorg/astria/pull/1121/files Adds storage of the celestia height into the DB, so we don't resync celestia data already read on restart. ci: update the test ci to run build/ci.go (#27) Previously, the test ci was running `go test -short ./...`. Instead run the build/ci.go to better test on the ci implement `AstriaMintableERC20`, update deposit tx to support ERC20 mints (#20) - implement `AstriaMintableERC20.sol`, allows an immutable address `bridge` to mint or burn - this address should be set at genesis and the sender of `DepositTx`s is set to this address - `DepositTx` was updated to include `to` and `data` which are set for erc20 mints (later on burns), `to` is set to the erc20 address of the `AstriaMintableERC20` and `data` is the calldata to call the mint function - genesis config fields for erc20s must be set to allow bridging erc20s from the sequencer. namely the erc20 contract address must be set, meaning the token must already be deployed testing: - i tested this manually end-to-end using the instructions in `contracts/README.md`. closes #9
SuperFluffy
pushed a commit
that referenced
this pull request
Apr 28, 2025
rebase on top of geth v1.13.6 Squash 916d6a4 to 3d160b2 3d160b2 is the last commit before the infamous squash + merge of geth into astria-geth The commit messages below are left unchanged for now so that they can be cleaned up later. Feature/grpc execution api (#1) * add buf config and generated grpc code * poc e2e grpc communication * comment out panicking code * logging help * now reads cli args. now stops grpc server on shutdown. * add mutex to GRPCServerHandler update readme containerize w/ github action to build and push to ghcr (#3) fix gh action syntax (#4) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax build and push manually (#5) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax * dont use github action, do it ourselves * push manually Feature/containerize (#6) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax * dont use github action, do it ourselves * push manually * correct multi line update SubmitTransaction to send tx to metro ethclient: ensure returned subscription is nil on error (#26976) core/state, trie: remove Try prefix in Trie accessors (#26975) This change renames StateTrie methods to remove the Try* prefix. We added the Trie methods with prefix 'Try' a long time ago, working around the problem that most existing methods of Trie did not return the database error. This weird naming convention has persisted until now. Co-authored-by: Gary Rong <garyrong0905@gmail.com> metrics/librato: ensure resp.body closed (#26969) This change ensures that we call Close on a http response body, in various places in the source code (mostly tests) core/vm: use atomic.Bool (#26951) Make use of new atomic types --------- Co-authored-by: Felix Lange <fjl@twurst.com> Co-authored-by: Martin Holst Swende <martin@swende.se> core/bloombits: use atomic type (#26993) core/state: use atomic.Bool (#26992) graphql: fix data races (#26965) Fixes multiple data races caused by the fact that resolving fields are done concurrently by the graphql library. It also enforces caching at the stateobject level for account fields. eth/tracers/native: prevent panic for LOG edge-cases (#26848) This PR fixes OOM panic in the callTracer as well as panicing on opcode validation errors (e.g. stack underflow) in callTracer and prestateTracer. Co-authored-by: Martin Holst Swende <martin@swende.se> internal/debug: add log.logfmt flag to set logging to use logfmt (#26970) docs: update outdated DeriveSha docs comment (#26968) add localnet genesis update metro-transactions dep Adding features to mempool to support our pre-ordered txs (#2) * Adding features to mempool to support our pre-ordered txs * Add framing for gRPC to execute blocks * Remove public engine API call * Fix circular dependencies * Updated to use new Init, and fill out starting attributes * Add clear astriaordered, cleanup * readme fix * add bash and jq to final docker image * make txpool interface * no more panics, update DoBlock to also update state + store block * cleanup * set post-merge at genesis * cleanup * doc update * remove txpool interface changes * cleanup * cleanup * build and push images wih tags defined by git tags * build for multiple architectures. use docker-metadata action for semver * add push: true * only build arm for git tags/releases --------- Co-authored-by: Jesse Snyder <jessetsnyder@gmail.com> Co-authored-by: elizabeth <elizabethjbinks@gmail.com> tag image with latest for builds from astria branch (#8) rename state_root to block_hash use FROM --platform=$BUILDPLATFORM to build arm images correctly (#9) use v4 of build and push action (#10) * use FROM --platform=$BUILDPLATFORM to build arm images correctly * use v4 of docker build and push fix arm builds. only build arm for tags and merges to default branch. (#12) * fix arm builds. only build arm for tags and merges to default branch. * fix gh action function syntax * more correct comment * build for semver tags Features and fixes needed for contract deployment test (#13) * add and remove tx from geth mempool so forge can deploy a contract * pass metro addr and port by flag * fixes from pr feedback * formatting implement FinalizeBlock gRPC call bump metro-transactions dep Update Protos (#16) * uses new protos * remove old protos update deps update deps no more unknown/unknown image (#19) * no more unknown/unknown image * hardcode condition Changing from prev_state_root to prev_block_hash remove metro integrate submission to sequencer log updates use env vars for chain id/tendermint endpoint rename to cometbft Make the environment variables actually work cleaner simple logging change removed sleep before getPayload by removing goroutine in buildPayload removed unneeded comment removed for loop from buildPayload Remove direct sequencer submission cleanup go.mod Migrate to the v1alpha1 API Missed pieces set genesis block as head/safe/final remove all mentions of metro Initial implementation of Execution v1alpha2 api Integrate updates remove need for consensus api Updates small updates Update grpc/execution/server.go Co-authored-by: noot <36753753+noot@users.noreply.github.com> logging remove unused dependency Mark the chain 'finalized' on startup updates txpool fixes, remove v1alpha1, use forkchoice some cleanup formatting and such cleanup think it's good add new fields to base example genesis Update gh action for release branch Add timestamp to block generated from header Add missing parent hash chore: add some metrics specifically around astria tx execution small refactor fix feat: everything needed for genesis fix review + metrics update to prod pushed buf fix after grpc update switch to new execution apis package feat: support new API utilizing rollup data update to latest proto update docker to login implement DepositTx type and handle in state transition and execute_block cleanup fixes to get deposits working, update genesis.json cleanup cleanup exit node if genesis astria fields are unset add bridge address and asset IDs to genesis and check during execution don't allow deposit or blob txs in sequenced data minimum changes required for fork (#2) Updates the codeowners and sets a new package repo for `astria-geth` repo. Once this is merged, I can push the astria specific tags and we will get built images for old tags on this repo. feat: update protos, config for fees, updated bridging (#1) Allow setting eip 1559 fee params (#3) Adds a per block height config to set some EIP1559 fee params (elasticity and denominator) + adds a min base fee parameter. collect the base fee instead of burning it (#5) EIP-1559 burns the base fee. This doesn't make sense for rollups. The set fee recipient should collect it. Chore: Add forkdiff gh-page (#14) we need a static webpage to show changes made to astria-geth fork, this should be used both internally and externally. using a [forkdiff](https://github.com/protolambda/forkdiff) tool to create a static webpage. add a fork.yaml file structs the changes view and define the commits to compare with pages.yaml workflow deploys to gh-pages Currently deploying on PR for testing. feat: remove txs which do not end up in a block out of the mempool (#15) When the txs we receive from `ExecuteBlocks` call fail to end up in a block, we keep them lying around the geth mempool but clear it from the astria mempool. This PR attempts to remove such txs from the geth mempool. chore: make server logs less noisy (#12) Moves the initial request logs to debug logs, reduces the amount of information printed, and ensures okay formatting. Logs from Geth were very messy, block hashes were messy bytes, and all the information from each request was logged out unnecessarily. fix: continue the tx validation loop if assetId length check fails (#18) Currently, we check if the length of the assetId sent by the rollup is of 32 bytes. If the check fails, we still continue executing the code which will lead to a panic as in the subsequent we try to index the bytes. The fix would be to ignore this tx when the validation fails fix: Fix tests which were failing (#16) This PR fixes all the failing tests in the astria-geth repo apart from the [`TestReimportMirroredState` ](https://github.com/astriaorg/astria-geth/blob/f5a95fe13d581ae1ec67caeaae6efe5338399a3e/consensus/clique/clique_test.go#L44) (More details at the end) The main reasons for the tests failing were: 1. #5 which transfers the basefee to the fee recipient rather than burning it. This caused all the tests which expect a certain state root to fail since they assume that the basefee would be burnt. It also failed tests which check for the balance of the fee recipient which fails because we also have to account for the base fee. To fix them, we replaced the tests with the expected state root and updated the tests with the expected balance of the fee recipient. 2. Picking txs from the `AstriaOrdered` rather than from the geth mempool. Lot of tests place the txs in the Geth mempool and they end up failing as we do not pick txx from their. To fix this, we had to add these txs to the `AstriaOrdered` mempool instead This PR also adds a github action to run the tests. `TestReimportMirroredState` is very tricky to fix since the fee recipient while building the block and while verifying the block before inserting onchain is different. This is because while verifying the block, we pick the fee recipient from the consensus engine's author which in this case is the signer of the block. It's tricker to fix it and we have commented it out for now. chore: migrate to depot to fix builds (#23) buildjet has not been working on this repo, and their customer support has been non responsive. feat: Add tests for execution grpc server and astria configs (#17) This PR adds the following: 1. Tests for astria execution grpc server 2. Tests for astria config 3. Abstraction of astria validation logic and unit tests for them. feat!: store celestia height updates (#22) Updates geth to work with proto changes here: https://github.com/astriaorg/astria/pull/1121/files Adds storage of the celestia height into the DB, so we don't resync celestia data already read on restart. ci: update the test ci to run build/ci.go (#27) Previously, the test ci was running `go test -short ./...`. Instead run the build/ci.go to better test on the ci implement `AstriaMintableERC20`, update deposit tx to support ERC20 mints (#20) - implement `AstriaMintableERC20.sol`, allows an immutable address `bridge` to mint or burn - this address should be set at genesis and the sender of `DepositTx`s is set to this address - `DepositTx` was updated to include `to` and `data` which are set for erc20 mints (later on burns), `to` is set to the erc20 address of the `AstriaMintableERC20` and `data` is the calldata to call the mint function - genesis config fields for erc20s must be set to allow bridging erc20s from the sequencer. namely the erc20 contract address must be set, meaning the token must already be deployed testing: - i tested this manually end-to-end using the instructions in `contracts/README.md`. closes #9
SuperFluffy
pushed a commit
that referenced
this pull request
Apr 28, 2025
rebase on top of geth v1.13.7 rebase on top of geth v1.13.6 Squash 916d6a4 to 3d160b2 3d160b2 is the last commit before the infamous squash + merge of geth into astria-geth The commit messages below are left unchanged for now so that they can be cleaned up later. Feature/grpc execution api (#1) * add buf config and generated grpc code * poc e2e grpc communication * comment out panicking code * logging help * now reads cli args. now stops grpc server on shutdown. * add mutex to GRPCServerHandler update readme containerize w/ github action to build and push to ghcr (#3) fix gh action syntax (#4) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax build and push manually (#5) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax * dont use github action, do it ourselves * push manually Feature/containerize (#6) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax * dont use github action, do it ourselves * push manually * correct multi line update SubmitTransaction to send tx to metro ethclient: ensure returned subscription is nil on error (#26976) core/state, trie: remove Try prefix in Trie accessors (#26975) This change renames StateTrie methods to remove the Try* prefix. We added the Trie methods with prefix 'Try' a long time ago, working around the problem that most existing methods of Trie did not return the database error. This weird naming convention has persisted until now. Co-authored-by: Gary Rong <garyrong0905@gmail.com> metrics/librato: ensure resp.body closed (#26969) This change ensures that we call Close on a http response body, in various places in the source code (mostly tests) core/vm: use atomic.Bool (#26951) Make use of new atomic types --------- Co-authored-by: Felix Lange <fjl@twurst.com> Co-authored-by: Martin Holst Swende <martin@swende.se> core/bloombits: use atomic type (#26993) core/state: use atomic.Bool (#26992) graphql: fix data races (#26965) Fixes multiple data races caused by the fact that resolving fields are done concurrently by the graphql library. It also enforces caching at the stateobject level for account fields. eth/tracers/native: prevent panic for LOG edge-cases (#26848) This PR fixes OOM panic in the callTracer as well as panicing on opcode validation errors (e.g. stack underflow) in callTracer and prestateTracer. Co-authored-by: Martin Holst Swende <martin@swende.se> internal/debug: add log.logfmt flag to set logging to use logfmt (#26970) docs: update outdated DeriveSha docs comment (#26968) add localnet genesis update metro-transactions dep Adding features to mempool to support our pre-ordered txs (#2) * Adding features to mempool to support our pre-ordered txs * Add framing for gRPC to execute blocks * Remove public engine API call * Fix circular dependencies * Updated to use new Init, and fill out starting attributes * Add clear astriaordered, cleanup * readme fix * add bash and jq to final docker image * make txpool interface * no more panics, update DoBlock to also update state + store block * cleanup * set post-merge at genesis * cleanup * doc update * remove txpool interface changes * cleanup * cleanup * build and push images wih tags defined by git tags * build for multiple architectures. use docker-metadata action for semver * add push: true * only build arm for git tags/releases --------- Co-authored-by: Jesse Snyder <jessetsnyder@gmail.com> Co-authored-by: elizabeth <elizabethjbinks@gmail.com> tag image with latest for builds from astria branch (#8) rename state_root to block_hash use FROM --platform=$BUILDPLATFORM to build arm images correctly (#9) use v4 of build and push action (#10) * use FROM --platform=$BUILDPLATFORM to build arm images correctly * use v4 of docker build and push fix arm builds. only build arm for tags and merges to default branch. (#12) * fix arm builds. only build arm for tags and merges to default branch. * fix gh action function syntax * more correct comment * build for semver tags Features and fixes needed for contract deployment test (#13) * add and remove tx from geth mempool so forge can deploy a contract * pass metro addr and port by flag * fixes from pr feedback * formatting implement FinalizeBlock gRPC call bump metro-transactions dep Update Protos (#16) * uses new protos * remove old protos update deps update deps no more unknown/unknown image (#19) * no more unknown/unknown image * hardcode condition Changing from prev_state_root to prev_block_hash remove metro integrate submission to sequencer log updates use env vars for chain id/tendermint endpoint rename to cometbft Make the environment variables actually work cleaner simple logging change removed sleep before getPayload by removing goroutine in buildPayload removed unneeded comment removed for loop from buildPayload Remove direct sequencer submission cleanup go.mod Migrate to the v1alpha1 API Missed pieces set genesis block as head/safe/final remove all mentions of metro Initial implementation of Execution v1alpha2 api Integrate updates remove need for consensus api Updates small updates Update grpc/execution/server.go Co-authored-by: noot <36753753+noot@users.noreply.github.com> logging remove unused dependency Mark the chain 'finalized' on startup updates txpool fixes, remove v1alpha1, use forkchoice some cleanup formatting and such cleanup think it's good add new fields to base example genesis Update gh action for release branch Add timestamp to block generated from header Add missing parent hash chore: add some metrics specifically around astria tx execution small refactor fix feat: everything needed for genesis fix review + metrics update to prod pushed buf fix after grpc update switch to new execution apis package feat: support new API utilizing rollup data update to latest proto update docker to login implement DepositTx type and handle in state transition and execute_block cleanup fixes to get deposits working, update genesis.json cleanup cleanup exit node if genesis astria fields are unset add bridge address and asset IDs to genesis and check during execution don't allow deposit or blob txs in sequenced data minimum changes required for fork (#2) Updates the codeowners and sets a new package repo for `astria-geth` repo. Once this is merged, I can push the astria specific tags and we will get built images for old tags on this repo. feat: update protos, config for fees, updated bridging (#1) Allow setting eip 1559 fee params (#3) Adds a per block height config to set some EIP1559 fee params (elasticity and denominator) + adds a min base fee parameter. collect the base fee instead of burning it (#5) EIP-1559 burns the base fee. This doesn't make sense for rollups. The set fee recipient should collect it. Chore: Add forkdiff gh-page (#14) we need a static webpage to show changes made to astria-geth fork, this should be used both internally and externally. using a [forkdiff](https://github.com/protolambda/forkdiff) tool to create a static webpage. add a fork.yaml file structs the changes view and define the commits to compare with pages.yaml workflow deploys to gh-pages Currently deploying on PR for testing. feat: remove txs which do not end up in a block out of the mempool (#15) When the txs we receive from `ExecuteBlocks` call fail to end up in a block, we keep them lying around the geth mempool but clear it from the astria mempool. This PR attempts to remove such txs from the geth mempool. chore: make server logs less noisy (#12) Moves the initial request logs to debug logs, reduces the amount of information printed, and ensures okay formatting. Logs from Geth were very messy, block hashes were messy bytes, and all the information from each request was logged out unnecessarily. fix: continue the tx validation loop if assetId length check fails (#18) Currently, we check if the length of the assetId sent by the rollup is of 32 bytes. If the check fails, we still continue executing the code which will lead to a panic as in the subsequent we try to index the bytes. The fix would be to ignore this tx when the validation fails fix: Fix tests which were failing (#16) This PR fixes all the failing tests in the astria-geth repo apart from the [`TestReimportMirroredState` ](https://github.com/astriaorg/astria-geth/blob/f5a95fe13d581ae1ec67caeaae6efe5338399a3e/consensus/clique/clique_test.go#L44) (More details at the end) The main reasons for the tests failing were: 1. #5 which transfers the basefee to the fee recipient rather than burning it. This caused all the tests which expect a certain state root to fail since they assume that the basefee would be burnt. It also failed tests which check for the balance of the fee recipient which fails because we also have to account for the base fee. To fix them, we replaced the tests with the expected state root and updated the tests with the expected balance of the fee recipient. 2. Picking txs from the `AstriaOrdered` rather than from the geth mempool. Lot of tests place the txs in the Geth mempool and they end up failing as we do not pick txx from their. To fix this, we had to add these txs to the `AstriaOrdered` mempool instead This PR also adds a github action to run the tests. `TestReimportMirroredState` is very tricky to fix since the fee recipient while building the block and while verifying the block before inserting onchain is different. This is because while verifying the block, we pick the fee recipient from the consensus engine's author which in this case is the signer of the block. It's tricker to fix it and we have commented it out for now. chore: migrate to depot to fix builds (#23) buildjet has not been working on this repo, and their customer support has been non responsive. feat: Add tests for execution grpc server and astria configs (#17) This PR adds the following: 1. Tests for astria execution grpc server 2. Tests for astria config 3. Abstraction of astria validation logic and unit tests for them. feat!: store celestia height updates (#22) Updates geth to work with proto changes here: https://github.com/astriaorg/astria/pull/1121/files Adds storage of the celestia height into the DB, so we don't resync celestia data already read on restart. ci: update the test ci to run build/ci.go (#27) Previously, the test ci was running `go test -short ./...`. Instead run the build/ci.go to better test on the ci implement `AstriaMintableERC20`, update deposit tx to support ERC20 mints (#20) - implement `AstriaMintableERC20.sol`, allows an immutable address `bridge` to mint or burn - this address should be set at genesis and the sender of `DepositTx`s is set to this address - `DepositTx` was updated to include `to` and `data` which are set for erc20 mints (later on burns), `to` is set to the erc20 address of the `AstriaMintableERC20` and `data` is the calldata to call the mint function - genesis config fields for erc20s must be set to allow bridging erc20s from the sequencer. namely the erc20 contract address must be set, meaning the token must already be deployed testing: - i tested this manually end-to-end using the instructions in `contracts/README.md`. closes #9
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When the txs we receive from
ExecuteBlockscall fail to end up in a block, we keep them lying around the geth mempool but clear it from the astria mempool.This PR attempts to remove such txs from the geth mempool.