Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

meta-tx gas cost estimation #8114

Closed
4 tasks done
Tracked by #8075
jakmeier opened this issue Nov 24, 2022 · 4 comments · Fixed by #8597
Closed
4 tasks done
Tracked by #8075

meta-tx gas cost estimation #8114

jakmeier opened this issue Nov 24, 2022 · 4 comments · Fixed by #8597
Assignees
Labels
A-params-estimator Area: runtime params estimator A-transaction-runtime Area: transaction runtime (transaction and receipts processing, state transition, etc) T-contract-runtime Team: issues relevant to the contract runtime team

Comments

@jakmeier
Copy link
Contributor

jakmeier commented Nov 24, 2022

Meta transactions are interesting for estimation. The key observations are:

  • The work done by meta transactions is the same as transaction to receipt conversion, with some extra checks.
  • Meta transaction adds work in 3 steps, as opposed to normal transactions that have only a send and an exec part. This affects all inner actions as well.

In the initial implementation we went with the following design:

  • At the relayer, send fees for all inner actions are used twice and burn once.
  • At the user shard, we burn the previously reserved send gas for all inner actions.
  • The final receipt is no longer distinguishable from a normal receipt. Logically, it is also charged normally.

The reaming work here is:

  • Figure out if the extra cost for delegating an action (pay the send fee once more) is problematic for some use cases.
  • Make sure the split between send / exec in all actions is not horribly wrong. (Today almost all are just split 50:50 but it didn't matter much, in meta-transactions it matters more.)
  • Define the exact numbers for the delegate action itself. The cost for creating new actions receipts should be taken as a base line.
  • Add tests that gas costs are exactly what is expected.
@jakmeier
Copy link
Contributor Author

jakmeier commented Dec 16, 2022

To figure out the cost split between send and exec costs, I I measured costs for action verification (=send cost) and action execution separately. This should be pretty accurate for CPU costs.

But disk and network costs aren't reflect well. Network costs and disk write are in completely separate parts of the code. And disk reads in this experiment have the same problem as any IO estimations have today. (See also #5995 and #8214)

Based on these imperfect measurements, here is a summary table for what the split between send and exec costs should be (instead of 50-50).

Action send (not sir) exec Send / total
Transfer 1545248000 2405930000 39.11%
Create Account 0 3628600000 0.00%
Delete Account 138354800000 758839800000 15.42%
Add Full Access Key 0 351170600000 0.00%
Add Function Call Key 1559262000 464488800000 0.33%
Delete Key 1584712000 ? (see missing data below) ? (1.6% of current param)
Stake 68130428000 1296908000 98.13% (1.2% of current param)

The good news is that most costs have less than 50% of the cost in send. This means it is safe to just charge the send cost twice.

The stake action is an outlier with almost all costs in send. Luckily, this gas parameter seems to be overcharged today. The estimation gave us 0.068 Tgas for send costs. But we currently charge 0.142 Tgas. So this should be safe as well. (Btw, the reason why exec cost is estimated so low is because the estimator uses a mocked epoch manager, this should be improved...)

Based on this, I would tentatively conclude that we should just charge 2x the send cost to ship meta transactions. We can maybe reduce costs later.

Missing data:

  • Delete key execution would require more implementation effort but is currently not needed. We already know that the send cost is only 1.6% of the current parameter.
  • Action receipt creation: Needs separate investigation
  • Data receipts: Needs separate investigation
  • Function call / deployment: Would require more implementation effort. But both should be more expensive on execution compared to validation.

jakmeier added a commit to jakmeier/nearcore that referenced this issue Dec 20, 2022
Add action cost estimations that measure send and exec separately in
isolation from all other costs.

These new costs do not replace existing estimations.
They are only used to inform the decision for send/exec gas split,
which today is almost always fifty-fifty. The new estiamtions show that
often times the send costs should be cheaper. But we have to be careful
before we reduce any costs because we unfortunately ignore network costs
in all our estimations. (See near#8214.)

The main purpose of the new estimations is to verify the gas cost
strategy for meta transactions. (See near#8114.) There, we execute the
send step twice, so we want to ensure this side of the action cost is
not undercharged. It seems to be overcharged at the moment, so we should
be safe to move forward with the proposed strategy.
@jakmeier
Copy link
Contributor Author

The last two point here are blocked on merging #7497 so I can actually not work on this right now.

near-bulldozer bot pushed a commit that referenced this issue Dec 22, 2022
Add action cost estimations that measure send and exec separately in
isolation from all other costs.

These new costs do not replace existing estimations.
They are only used to inform the decision for send/exec gas split,
which today is almost always fifty-fifty. The new estiamtions show that
often times the send costs should be cheaper. But we have to be careful
before we reduce any costs because we unfortunately ignore network costs
in all our estimations. (See #8214.)

The main purpose of the new estimations is to verify the gas cost
strategy for meta transactions. (See #8114.) There, we execute the
send step twice, so we want to ensure this side of the action cost is
not undercharged. It seems to be overcharged at the moment, so we should
be safe to move forward with the proposed strategy.
nikurt pushed a commit to nikurt/nearcore that referenced this issue Dec 23, 2022
Add action cost estimations that measure send and exec separately in
isolation from all other costs.

These new costs do not replace existing estimations.
They are only used to inform the decision for send/exec gas split,
which today is almost always fifty-fifty. The new estiamtions show that
often times the send costs should be cheaper. But we have to be careful
before we reduce any costs because we unfortunately ignore network costs
in all our estimations. (See near#8214.)

The main purpose of the new estimations is to verify the gas cost
strategy for meta transactions. (See near#8114.) There, we execute the
send step twice, so we want to ensure this side of the action cost is
not undercharged. It seems to be overcharged at the moment, so we should
be safe to move forward with the proposed strategy.
@jakmeier
Copy link
Contributor Author

The point with exact gas cost tests is covered in one of my PRs for integration tests. (#8559)

The final point, defining the cost of the delegate action itself, is still open. My current thinking is:

  • I expect this can be a small cost. It's basically only the cost of the inner signature validation and other checks on the delegate action fields.
  • The exact signature validation logic is still undecided. We need at least a first draft of Change signed delegate action format to avoid confusing it with other things that might be signed by an account key #8271 to materialize, or else an accurate estimation doesn't make sense.
  • It's not ideal but if necessary, I can make a ball-park estimation on today's implementation. If it is really as small as I think it will, then we can also set it to 10x the current estimate and be fairly confident it will be a safe cost even if we change the signature scheme. And we could keep the cost even after we finalize the code, as the cost would still be cheap enough that nobody would bother.

@jakmeier
Copy link
Contributor Author

Base gas cost estimations

Approach 2 + 3 are based on the code here: #8596

Approach 1: Theory

SEND: Validation only checks that there is only one delegate action. (Validating inner actions is charged separately from the base.)
That's basically one if condition, it can be seen as a zero gas cost.

EXEC: Here the actual checks happen, to ensure the meta transaction is valid.

  • The most expensive part is looking up keys in the DB. But that part is paid for in the create_action_receipt cost and doesn't need to be accounted for in the delegate action base cost.
  • The extra verification is basically just a signature verification of a 32 byte hash plus the construction of that hash.
    • A meta transaction could become quite large if the inner actions are large. That would make the hash expensive. However, the bytes in each inner action is already paid for. We charge 2x send and 1x exec costs for the inner actions upfront. The first send cost is for normal transaction verification, the second is exactly for this additional check. The per-byte cost should thus be covered and we should only worry about the constant overhead that a delegate action adds.
    • The signature verification should cost the same as wasm_ed25519_verify_base + 32 * wasm_ed25519_verify_byte = 0.21 Tgas

=> Approach 1 suggests 0 gas send / 210 Ggas exec

Approach 2: Measure an empty delegate action in the traditional estimator framework

In this approach, the empty delegate action cost is measured and the action creation cost is subtracted from the result.

This yields extreme volatility, because both costs move in the range of 1ms.
Subtracting the two numbers is basically pointless, since the expected difference is much smaller than the variance in this approach.

But we can fall back to QEMU based measurements. They aren't as accurate but more stable. Doing this, I get 540k - 580k x86 instructions. Using our 8 instructions / ns norm, this translates to 67.5 Ggas - 72.5 Ggas.

Following old practices, we multiply the higher value by 3 for extra safety (=217.5 Ggas) and divide the result evenly between send and exec costs. (= 108.75 Ggas + 108.75 Ggas)

Approach 3: Measure exec and send costs separately in estimator framework

SEND: Again, high volatility, usually betwenn 0 and 50us. But it looks like just measuring noise.
This is in line with the theory, as this should basically be for free.

EXEC: 71 - 72us with very little variance. Using the usual 3x safety multiplier, we would end up with 216 Ggas

Summary

send exec
Approach 1 0 210 Ggas
Approach 2 108.75 Ggas 108.75 Ggas
Approach 3 0 gas 216 Ggas

Using 3 completely different approaches, I ended up at a total cost of around 210 Ggas each time.
But approach 1 and 3 found that all cost is due to the execution part, whereas the send part is free.

Overall I suggest to go with 200 Ggas in each parameter, knowing that this is probably higher than necessary. But it is still about 11x cheaper than the place holder value.

action_delegate: {
  send_sir: 200_000_000_000,
  send_not_sir: 200_000_000_000,
  execution: 200_000_000_000,
}

This should also give us some headroom if we need to do more checks on the sender side in the future.

jakmeier added a commit to jakmeier/nearcore that referenced this issue Feb 17, 2023
for estimation data, please refer to
near#8114 (comment)
@jakmeier jakmeier linked a pull request Feb 17, 2023 that will close this issue
@jakmeier jakmeier added A-transaction-runtime Area: transaction runtime (transaction and receipts processing, state transition, etc) T-contract-runtime Team: issues relevant to the contract runtime team A-params-estimator Area: runtime params estimator labels Feb 17, 2023
near-bulldozer bot pushed a commit that referenced this issue Feb 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-params-estimator Area: runtime params estimator A-transaction-runtime Area: transaction runtime (transaction and receipts processing, state transition, etc) T-contract-runtime Team: issues relevant to the contract runtime team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant