fix: Outbox replay state management, saves ~15k gas per msg

[shotaronowhere]

I implemented a simple packed bitmap to prevent message replay.

The original implementation uses a mapping to a boolean which wastes 255 bits of state per message. By packing the spent boolean into a bitmap, only one zero -> nonzero SSTORE call is required ~20k gas.

Since the packed bitmap is set to non-zero for the first message, all 255 subsequent messages only cost ~5k gast since non-zero -> non-zero SSTORE calls cost ~5k gas.

Hence a ~15k gas savings per message.

[tsahee]

Let's use 255 instead of 256 here.. this will leave the most significant bit reserved.
We might do something clever with this reserved bit later, to avoid 1/256 users paying more gas for the release.

[shotaronowhere]

Sure, just made that change. : )

[shotaronowhere]

It goes without saying, but message relaying is most efficient when batched into groups of 255 (or 256 including the MSB in the replay bitmap). Specifically batching multiple messages index_initial to index_final is most efficient when index_initial % 255 == 0 and (index_final - index_initial) % 254 == 0. e.g indexes 0 to 254, or 255 to 509, etc.

since SSTORE only costs 100 gas since the spent[index] slot is a "dirty slot", already updated in current execution context, hence subsequent SSTORES in the batch will be cheaper than ~5k gas (if each message is relayed in individual transactions). Optimal batching doesn't require any contract level changes. A proxy contract can batch transactions. Hopefully the sequencer already prioritizes batching together ARB_SYS.sendTxToL1
calls which were submitted originally in sequential grouped batches. Then the L2 message sender can optimize this batching.

Replay "Spent" State Management Gas Summary

• Original relaying 256 messages would incur 256*20000 = 5 million gas in SSTORE state modifying overhead.
• Replay spent boolean bitmap packing relaying 256 messages would incur 20000 + 255*5000= 1 million gas in SSTORE state modifying overhead.
• Replay spent boolean bitmap packing & Optimal Batching relaying 256 messages would incur 20000 + 255*100= 45k gas in SSTORE state modifying overhead.

Moreover, creating an auxiliary function specifically for executing batch transactions which checks the spent bitmap only once, and sets it only once, before sequentially executing each message (check effects pattern), can bring this down the state overhead to about 20k (no 100 gas SSTORE calls). If this doesn't sound like too much complexity (in smart contract dev, KISS simplicity is often the best, right?) I could make an implementation.

This might be overkill, but in the future, I expect state cost on L1 to be very expensive until sharding and data availability arrives (2030? who knows). EIP 5022 doesn't have much discussion, but I completely support it, SSTORE should actually be more expensive. With multidimensional gas pricing storage will probably be at a premium.

[greenlucid]

Crazy savings. But, you missed on the SLOADs that are also storage overhead.
Each first SLOAD is 2100 gas, and each hot SLOAD is 100 gas. SSTORE on hot keys costs 2100 less gas. I'll do the math with 256 messages per bitmap:

So actually storage overhead costs are:

• Previous cost 256*22100 = 5.6M gas | 22100 / msg
• Bitmap, trigger on different messages cost 22100 + 255*5000 = 1.3M gas | 5066 / msg
• Bitmap, all triggers on the same batched message 22100 + 255*200 = 73.1k gas | 286 / msg

Bitmap + batch is a 76x performance boost to previous cost, and 17x to bitmap + different triggers

Moreover, creating an auxiliary function specifically for executing batch transactions which checks the spent bitmap only once, and sets it only once, before sequentially executing each message (check effects pattern), can bring this down the state overhead to about 20k (no 100 gas SSTORE calls). If this doesn't sound like too much complexity (in smart contract dev, KISS simplicity is often the best, right?) I could make an implementation.

It's true that it would be just 22k in state overhead, due to triggering SSTORE once, but it may be the case that the extra non-state computation needed to make it work are over the 51k gas that you would be saving. Also, take into account that only applies to the most optimistic scenario. In more realistic scenarios, this process would occur multiple times, and then, the simple bitmap solution may be more efficient.

Why it's realistic to assume multiple calls:

• the gas cost of triggering all the messages of the bitmap is over the gas limit (likely, especially if some messages are expensive to trigger)
• a bitmap contains messages from distinct parties and they can't, or don't want to coordinate to execute the messages in the same transaction
• even if the parties involved know that there's no MEV protection, they may want to trigger the messages in a different order

[greenlucid]

@tsahee

Let's use 255 instead of 256 here.. this will leave the most significant bit reserved.
We might do something clever with this reserved bit later, to avoid 1/256 users paying more gas for the release.

Since there's the possibility of MEV, plus the fact that the first party that "opens" this bitmap is still getting something out of it (getting to be first, value of relaying the msg is probably higher than the cost), and the first solution taking this cost into account for every message, I'm not sure how good reserving this bit is.
Would you be planning on subsidizing this initial SSTORE?

[cla-bot]

We require contributors to sign our Contributor License Agreement. In order for us to review and merge your code, please sign the linked documents below to get yourself added. https://na3.docusign.net/Member/PowerFormSigning.aspx?PowerFormId=b15c81cc-b5ea-42a6-9107-3992526f2898&env=na3&acct=6e152afc-6284-44af-a4c1-d8ef291db402&v=2

[fredlacs]

@cla-bot check

[cla-bot]

The cla-bot has been summoned, and re-checked this pull request!

[codecov]

Codecov Report

Merging #623 (def5990) into master (ef1ea30) will decrease coverage by 4.51%.
The diff coverage is 100.00%.

❗ Current head def5990 differs from pull request most recent head 31b7910. Consider uploading reports for the commit 31b7910 to get more accurate results

@@            Coverage Diff             @@
##           master     #623      +/-   ##
==========================================
- Coverage   52.02%   47.50%   -4.52%     
==========================================
  Files         225      214      -11     
  Lines       26421    22394    -4027     
  Branches      496      496              
==========================================
- Hits        13745    10638    -3107     
+ Misses      11285    10366     -919     
+ Partials     1391     1390       -1     

[tsahee]

@greenlucid
I'd assume many redeems are not timing-critical, since you already agree to wait a week. Putting much more cost on the first of every 256 doesn't seem optimal.
Subsidizing / socializing the initial SSTORE in some way is certainly reasonable. e.g. it could be done as part of assertion posting. Fees to cover it might be collected in some fair way on L2.. there are many interesting options here. The nice thing is that we don't have to choose now. Keeping one bit reserved has a low overhead - and it opens up many possibilities which we could choose from later.

[gzeoneth]

@cla-bot check

[cla-bot]

We require contributors to sign our Contributor License Agreement. In order for us to review and merge your code, please sign the linked documents below to get yourself added. https://na3.docusign.net/Member/PowerFormSigning.aspx?PowerFormId=b15c81cc-b5ea-42a6-9107-3992526f2898&env=na3&acct=6e152afc-6284-44af-a4c1-d8ef291db402&v=2

[cla-bot]

The cla-bot has been summoned, and re-checked this pull request!

[cla-bot]

We require contributors to sign our Contributor License Agreement. In order for us to review and merge your code, please sign the linked documents below to get yourself added. https://na3.docusign.net/Member/PowerFormSigning.aspx?PowerFormId=b15c81cc-b5ea-42a6-9107-3992526f2898&env=na3&acct=6e152afc-6284-44af-a4c1-d8ef291db402&v=2

[hkalodner]

Hey @shotaronowhere, appreciate the contributions and we'd love to get these merged. I saw you signed the arbitrum CLA, but we actually have a new one for this repo. Would appreciate if you could sign the CLA for this repo when you get a chance 😄

[hkalodner]

@cla-bot check

[cla-bot]

The cla-bot has been summoned, and re-checked this pull request!

[shotaronowhere]

By the way, as I take it, the plan is that Offchain Labs can subsidize opening the slots by setting the MSB of a large set of replay bitmaps in advance at a time when gas is extremely cheap. You can then socialize the cost amongst all the message senders, hope fully without any rent seeking, but hey, that's a business model, need to keep the lights on some how. The coordinator fee.

[PlasmaPower]

We don't have a precise plan for how we're going to set the MSBs in advance (and we may not launch with it). It's a low overhead change though, so our thinking is that it's better to keep our options open by reserving it.

[PlasmaPower]

Thanks! LGTM