Add SIMD-0017: Priced Compute Units

[anoushk1234]

Problem
All transactions are required to pay the same base fee irrespective of the amount of compute units they use. This a spam/ddos vector and can cause blocks to reach their compute limit very frequently.
More on the issue here solana-labs/solana#29492

Solution
Transactions should be priced based on the amount of compute units they use instead of a constant base fee.

[jacobcreech]

Hello @anoushk1234! I love the enthusiasm with creating a new proposal, but this document does not follow the format designated in SIMD-0001. I'd recommend taking this idea into #core-technology on discord and vetting it out first to understand the likelihood of it being accepted, as specified in the idea stage of SIMD-0001.

[Overclock-Validator]

Hello @anoushk1234! I love the enthusiasm with creating a new proposal, but this document does not follow the format designated in SIMD-0001. I'd recommend taking this idea into #core-technology on discord and vetting it out first to understand the likelihood of it being accepted, as specified in the idea stage of SIMD-0001.

Hey Jacob, yeah that sounds good to us. Anatoly DM'd me asking to write it up haha so that's why we're posting. The fee per cu vote normalization was his idea and I passed things along to Anoush.

[anoushk1234]

Hello @anoushk1234! I love the enthusiasm with creating a new proposal, but this document does not follow the format designated in SIMD-0001. I'd recommend taking this idea into #core-technology on discord and vetting it out first to understand the likelihood of it being accepted, as specified in the idea stage of SIMD-0001.

Regarding the idea we're already discussing this with Anatoly and other validators in dms, for the format I was slightly confused since some proposals didn't follow the exact same format so i assumed it was flexible. Nevertheless we will improve it, thanks.

[Overclock-Validator]

For some more context, the voting fee normalized pricing (0.000005 Sol/total vote cu's) mentioned would cost:

2100 cu vote transaction = 0.000005 Sol so 2.38 * 10^-9 per cu
-At $15 Sol this is 0.0075 cents per vote and ~$3,500 per year in voting costs as per Cogent's calculator
-At $50 Sol this is 0.025 cents per vote and ~$11,700 per year in voting costs
https://cogentcrypto.io/ValidatorProfitCalculator

For 1 year costs the calculation used is:
Blocks per year = (1000ms/400ms per slot) * 60 sec * 60 min * 24 hours * 365 days = 78,840,000 slots

1k cu = 0.000002380952 Sol
-At $15 Sol this is ~0.00357 cents or ~$2,816 per year if landed each block
-At $50 Sol this is ~0.0119 cents or $9,386 per year if landed each block

10k cu = 0.00002380952 Sol
-At $15 Sol this is ~0.0357 cents or ~$28,160 per year if landed each block
-At $50 Sol this is ~0.119 cents or $93,860 per year if landed each block

100k cu = 0.0002380952 Sol
-At $15 Sol this is ~0.357 cents or ~$281,600 per year if landed each block
-At $50 Sol this is ~1.19 cents or $938,600 per year if landed each block

1 million cu = 0.002380952 Sol
-At $15 Sol this is ~3.57 cents or ~$2,816,000 per year if landed each block
-At $50 Sol this is ~11.9 cents or $9,386,000 per year if landed each block

Network non-vote fees per year using non vote cu's of half the current block size (48 million cu is full block, so 24 mil cu):
-At $15 it's $67,577,142
-At $50 it's $225,257,142

Network vote fees per year using vote cu's equivalent to 2000 validators (2100 per vote * 2000 = 4.2 million cu per block):
-At $15 it's $11,826,000 paid by validators, with half of that burned and half of that going back to validators (mainly larger ones)
-At $50 it's $78,840,000 paid by validators to network burn and other validators

For reference, assuming slot times are 400ms (often slower than this on avg), our validator which has ~800k Sol stake and is around ~0.2% of stake would earn about 4500 Sol in block rewards just from non vote transactions with this current base fee per cu and assuming non vote cu's take up about 24 million cu per block.

[aeyakovenko]

for dapps, this would be closer to reality
lamports per CU = 0.000005 / 200_000

Since the standard transaction was 200k CUs. The maximum tx would then be 8x of the standard one. Vote fees would reduce. Because vote fees would reduce the network needs to guard more against vote spam. Easy way would be to only allow the top X validators that fill up to 33% of the compute budget.

[TheEnigmaa]

Looks like most of the main points have been covered here already.
My take:

• Vote tx CU is very small. If we scale everything to a vote tx we will be unnecessarily imposing egregious SOL costs to the "Standard" tx, which will negatively impact several protocols, especially CLOBS. Imo, the majority of these costs should be decided by the fee market. I agree that cheap 1.4m CU txs are a potential ddos vector and should be penalized more.
• As Toly mentioned above, I think 200k CU is a much more reasonable baseline, as this is what most early protocols had in mind anyways.
• I'm also in favor of reducing vote costs. This is a major barrier to smaller operators and overall decentralization of the network. I would be interested to hear proposals for reducing vote spam -- surely there must be better options than just increasing fees.
• One of the visions we've always had for Solana is create a network on which efficient markets can be built. By imposing an extremely high baseline fee. Market-Makers (and crankers on Openbook) will suffer.

[Overclock-Validator]

for dapps, this would be closer to reality lamports per CU = 0.000005 / 200_000

Since the standard transaction was 200k CUs. The maximum tx would then be 8x of the standard one. Vote fees would reduce. Because vote fees would reduce the network needs to guard more against vote spam. Easy way would be to only allow the top X validators that fill up to 33% of the compute budget.

I don't think a base fee per cu of 0.000005/200k cu will achieve any of the desired spam resistance and cause more trouble than it's worth:
-Most of the MEV transactions filling up blockspace are 200k to 600k cu or so at the moment and this proposed fee per cu is basically no disincentive for them to not continue to fill up blocks, leading to CLOBs, etc needing to guess priority fees during times like Bonk anyway.
-Validators would lose a solid source of stable revenue during bear market times. It's likely Solana will just get more FUD if network revenue is even lower. It'll also be harder to attract solid validator operators and maintain a reasonably sized validator set as inflation goes down and if we get a bear market again. We also want good hardware and not potatoes -- foundation validators are barely profitable and go as cheap as possible.
-The timely vote credit system and the impact of making voting fees mostly disappear should be thought through more -- there is an incentive for validators to drop votes because it will earn them a higher apy which not only allows them to earn more, but gain more stake. It would be easy for large validators to really screw and target the rankings of lower staked validators.
-Should voting fees exist as some barrier to entry? I do not think they should be extremely expensive obviously, but voting fees do prevent people from testing junk on mainnetbeta. For example, we tested a second server in Tokyo before moving there on mainnetbeta but did so as fast as possible due to the voting fees. If voting fees are tooo low I imagine you'll get people more willing to test voting mods and random things on mainnetbeta, perhaps using junk hardware. Timely vote credit system will cause potato validators to lose more rewards but it might be good if voting costs are just high enough to act as some deterrent.
-Is a dynamic base fee still planned? If 0.000005/200k fee per cu were to happen blocks will still be full anyway right now but congestion fee will rise considerably to what I'm not sure.

Why not something less disruptive and more spam-resistant such as 5x to 10x lower than your original suggestion? 10x lower would be 0.000005/20k cu which would leave validator base block rewards nearly the same as they are now assuming half full blocks. CLOB transactions would also be similarly priced afaik? Priority will be needed anyway if blocks become full and people tend to over pay in first price auctions, which was the motivation for EIP1559: "However, first-price auction may require users to either overpay transaction fees or have a long waiting time. In particular, in periods of high demand, transaction fees can be volatile due to which it is difficult for users to make the right bid. If we overbid, we end up overpaying. If we bid low, we may experience a long waiting time."

"Because vote fees would reduce the network needs to guard more against vote spam. Easy way would be to only allow the top X validators that fill up to 33% of the compute budget."
-I still think vote and non-vote cu's should be buffered from each other. Too many non-vote transactions potentially cause vote transactions to be dropped at the moment once the block compute limit is hit. I also still don't understand why votes shouldn't just have their own pricing given their very different functional purpose.

[betaconger]

@Overclock-Validator excellent thoughts, Here's my short take after reading all this:

Solana should go full EIP1559.

Here's my long take:

We're trying to tackle a few distinct issues:

1. Low validator rewards in low usage
2. Vote fees are "too high", related to issue 1
3. Low cost, high CUs spamming the network
4. "Optimizing" fees, whatever that means. Probably it means generating the most revenue.

There is one additional, very key consideration that has come to light recently: blocks on the network are actually filling up! I think there was a perception that existed even as recently as two weeks ago that Solana had functionally unlimited block space. This was based on the claim of tens of thousands (or soon millions) of transactions per second. This perception overlooked that a small number of very computationally intensive transactions can consume all available block computing power. The problem of how to extract fees from a market with a perceived infinite supply is much, much different than a problem where, all-of-a-sudden, supply is bound.

In light of bound supply, it makes sense to set base fees fairly low. I don’t know if it’s as low as .000005/200k, but the point is that, if blocks fill up relatively easily, this won’t matter. If lowering the base fee makes it easier for validators to survive in low usage times, then this is good. This addresses issues 1 and 2.

Priority fees should, in theory, address 3. However, we could still see that people take advantage of the system because they are using a huge amount of compute power and paying relatively little for it (compared to someone doing a simple transfer). This should be enough of an argument alone to say that fees and therefore priority fees, should be priced in CU. A 200k CU tx that pays .000005 + .00000001 should be beaten out by a 2k cu tx that also pays .000005 + .00000001. The 200k cu should have to pay more than 100* ( .000005 + .00000001) in order to beat out the simple transfer.

On to 4. @overlock-validator mentions the auction problem. EIP1559 solves this elegantly by rapidly raising the base fee, while still allowing for priority fees. The base fee gives the market an excellent idea of what is required to be paid in order to get your tx through in a reasonable time. This is fantastic. It reacts to demand and is informational. Base + Tip (priority) is an excellent (and now proven) model. Burn/give to validators however the community decides.

If EIP1559 is working great for ETH, and Solana now clearly has similar limits on supply, the same concept should work well for Solana.

Appendix:

When firedancer comes around, or whenever the hardware on the network can actually sustain much greater than 48mm CU, governance can simply vote to raise this limit (expand supply)

[hydrogenbond007]

Hey for stopping spam and hitting the block limit #16 would be a better choice as it would also incentivize the dapp developers rather than just Validators, also most of the spam is received by a few programs that become unusable for the regular user because of the cu limit hit by arbitrages that fail for the most part.
yea but the point on vote txns makes sense.

[tao-stones]

Application Fee #16 would help, especially to disincentivize spamming on hot accounts. But charging base fee based on transaction's requested CUs would help on preventing block space being filled up unnecessarily.

Priority fees should, in theory, address 3.

Agree. but it doesn't seem to do much because it is still too cheap. What do you guys think to increase priority fee from current micro-lamport per cu to lamport per cu? (note: vote transaction does not have priority fee, increasing priority fee rate will not change vote fee) Dynamic base fee #4 can build on top of this.

[hydrogenbond007]

Charging a base fee would affect all the users and I think the main focus here is reducing spam on specific mev hit accounts that fill the most block space. There are a bunch of SIMDs currently trying to solve similar problems #4 (comment) (Dynamic fee) #19 (Priced cu) #16 (app fee) would be better if we could have more clarity on this.

[tao-stones]

Application fee (#16) is to deincentivize "bad actors" to lock-up specific dApp accounts, the fee only applies to users use these dapps.

Dynamic base fee (#4) and this (Priced CU #19) are both trying to make base fee to better reflect the cost of executing a transaction. imho, this proposal (#19) is part of overarching design (#4), which can be tackled separately, and hopefully easily.

[apfitzge]

How would it look if fees were kept at 1.0 for all transactions and only increase for those above 88% block utilization? From a user perspective its feels pretty good being able to explain a flat fee and I'm sure there would be lots of questions if the fee is constantly changing.

Then validators get to choose which transactions have to pay higher fees by deciding which ones to defer and which ones to schedule early, which would be an example of what you say would cause lots of questions -- transactions which had higher fees just randomly because validators decided to defer them.

There's no enforced ordering of transactions within a block, except when txs conflict. So I'd say this idea is a non-starter with current PoH protocol.

[bji]

How would it look if fees were kept at 1.0 for all transactions and only increase for those above 88% block utilization? From a user perspective its feels pretty good being able to explain a flat fee and I'm sure there would be lots of questions if the fee is constantly changing.

Then validators get to choose which transactions have to pay higher fees by deciding which ones to defer and which ones to schedule early, which would be an example of what you say would cause lots of questions -- transactions which had higher fees just randomly because validators decided to defer them.

There's no enforced ordering of transactions within a block, except when txs conflict. So I'd say this idea is a non-starter with current PoH protocol.

I think you meant to reply to the comment I replied to?

[apfitzge]

I think you meant to reply to the comment I replied to?

yes

[R-K-H]

I think I recall @ripatel-fd talking about some better understanding of demand side due to contention within the scheduler of FD? As this seems to be a lagging response is there a way to smooth the curve out to ensure there's a more palatable experience from the UX?

I think I'm also stuck on what the "ideal" configuration is within the network, as it seems there are many different "best" interests at play here too. If validators are aiming to maximize rewards and users are aiming to reduce their txn fees, and dApps are left to manage their own access fees is there something that satisfies all parties?

Maybe a practical example might help, if I'm a market maker on Solana using Phoenix, how much more would I be expected to pay? Is that > then amount I'm willing to continue to be a participant in this economic activity?

Broadly, I agree that something should be done with fees and more optimal strategies are well worth the time to explore, I suppose I'm curious how this addresses DDoS and Spam with keeping the current stakeholders comfy.

[tao-stones]

@taozhu-chicago I don't get the motivation to keep the price high once the load drops below 88%. Seems like it should just start dropping as long as load is below the max capacity.

Oh, I see this sentence else Compute Unit price *= 1.125 or 0.875 until rest at initial value; isn't clear. As soon as utilization drops below 88%, it starts to * 0.875 till compute-unit-price at its initial value (eg 1); conversely, as soon as utilization goes above 74%, price starts to *= 1.125 till at its initial value.

it also seems like 88% is too low.
88% is 2 standard deviations away from average, which is already smoothed out by ema.

How much load do votes currently take up?

In last 7 days, vote takes about ~35% of block CUs

[brianlong]

One of the problems we have seen is that the priority fees do not reliably result in higher priority. In a competitive environment, a low-fee TX will sometimes be processed ahead of another TX with a higher priority fee. Has this been addressed/fixed yet? There might be confounding factors outside the fee structure that should be addressed before we can confidently optimize fees. (People won't use the fees if they don't work).

[apfitzge]

Replying to @bji's comment

I do not support this proposal.

---

The priority fee is expressed as "compute unit price", which is a confusing way to declare it; but sufficient. The total priority fee is "maximum declared CU * compute unit price".

I wonder what is found to be confusing about the terminology here. I'm been too involved with the terminology, so I probably don't have a good outside perspective. If you've got a suggestion on how to make it less confusing, I'd be happy to hear.

This SIMD also takes away the ability of the network to perform "public good" by providing very low cost-per-CU transaction processing when the network is highly under-utilized.

If the network is under-utilized this dynamic price would be low, and thus the cost-per-CU would be low. As far as I am aware, using the numbers @taozhu-chicago ran, the cost of the vast majority of transactions during these periods would be lower than the current static-fee approach.

Blocks should not reach compute limit by being filled by transactions that under-pay for their CU used. If that happened, it would be because validators are not properly scheduling transactions for maximum fees. If validators were scheduling transactions for maximum fees, they would highly de-prioritize high-CU transactions that pay low priority fees, to the extent that such transactions would only be included in blocks when "there is nothing better to do with the block space", i.e. no transactions have been submitted that pay more per CU.

Solana's current scheduler does prioritize by price-per-cu, if the cu_price is set.
Though I think we could probably do better in terms of resolving priorities for equal cu_price txs.

---

I think a counter-argument to the current approach is that there is not an automated response to demand.
From what I've seen, bots and users are not constantly monitoring recent priority fees nor adjusting their bids accordingly - they just send with a static priority that's good enough.
@taozhu-chicago, I think this might have only recently become feasible through some new RPC calls?
So perhaps we just haven't seen the changes in behavior yet.

Even if we saw more adoption of the current priority fee market, it's still a reactive process for users and bots.
By the time they've seen the minimum priority fee has increased for a previous block, the demand may have already dropped.
The slow potential for reaction is probably not "punishing" enough for malicious txs that are just trying to eat up blockspace for a short period of time.

I think if we were starting Solana today, there's very little chance we'd have a static base fee that was not dependent on the cost of execution for a transaction. The base-fees in this proposal are a replacement for that, not priority markets.

I don't mean to voice explicit support for the proposal at this point, just trying to give some thoughts as feedback. I'm sure @taozhu-chicago has some thoughts as well!

[apfitzge]

One of the problems we have seen is that the priority fees do not reliably result in higher priority. In a competitive environment, a low-fee TX will sometimes be processed ahead of another TX with a higher priority fee. Has this been addressed/fixed yet? There might be confounding factors outside the fee structure that should be addressed before we can confidently optimize fees. (People won't use the fees if they don't work).

A leader will begin packing at the beginning of their slot(s). If a low-priority transaction is there, but the high-priority transaction hasn't arrived then low-priority one would probably get packed before it.

Non-conflicting transactions are processed & packed in parallel. So even if both are within the leader's buffer, a low-priority tx could come before a high-priority one if they didn't conflict, but the high-priority was blocked by some other even higher priority tx.

[bji]

Replying to @bji's comment

I do not support this proposal.

The priority fee is expressed as "compute unit price", which is a confusing way to declare it; but sufficient. The total priority fee is "maximum declared CU * compute unit price".

I wonder what is found to be confusing about the terminology here. I'm been too involved with the terminology, so I probably don't have a good outside perspective. If you've got a suggestion on how to make it less confusing, I'd be happy to hear.

Because I think that a priority fee is a whole value that shouldn't be expressed as a function of CU. It should just be expressed in its own terms, as an absolute value, not tied to CU. A validator should be looking at many factors of a transaction, including:

• Requested CU
• Write locks
• Read locks
• Historical data about CU used by similar transactions
• Anything else that the validator could use to estimate what the cost of executing the tx will be relative to other tx

And then turning all that into an "expected total resource cost" for executing the tx. Then dividing this by priority fee + base_fee (which I believe at this time is only N_SIGNATURES*5000 lamports), to get a "total cost : total fees" ratio. Then priority ordering tx to get the maximum fees for minimum execution cost.

Having priority fees as a function of CUs obfuscates that fact that the function of priority fees, it makes it seem like they are somehow related to CUs, when in fact they are not, at least not any more than they are related to a whole host of other transaction factors.

This SIMD also takes away the ability of the network to perform "public good" by providing very low cost-per-CU transaction processing when the network is highly under-utilized.

If the network is under-utilized this dynamic price would be low, and thus the cost-per-CU would be low. As far as I am aware, using the numbers @taozhu-chicago ran, the cost of the vast majority of transactions during these periods would be lower than the current static-fee approach.

But the SIMD proposal did not include dynamic fees, it includes a fixed additional fee (for V2 it proposes governance for setting the fee, which still is not dynamic). Unless I'm missing something?

Blocks should not reach compute limit by being filled by transactions that under-pay for their CU used. If that happened, it would be because validators are not properly scheduling transactions for maximum fees. If validators were scheduling transactions for maximum fees, they would highly de-prioritize high-CU transactions that pay low priority fees, to the extent that such transactions would only be included in blocks when "there is nothing better to do with the block space", i.e. no transactions have been submitted that pay more per CU.

Solana's current scheduler does prioritize by price-per-cu, if the cu_price is set. Though I think we could probably do better in terms of resolving priorities for equal cu_price txs.

OK then, problem already solved? :)

I think a counter-argument to the current approach is that there is not an automated response to demand. From what I've seen, bots and users are not constantly monitoring recent priority fees nor adjusting their bids accordingly - they just send with a static priority that's good enough. @taozhu-chicago, I think this might have only recently become feasible through some new RPC calls? So perhaps we just haven't seen the changes in behavior yet.

Agreed, this is an important part of the system that is missing, but I don't think it's addressed by this SIMD proposal.

Even if we saw more adoption of the current priority fee market, it's still a reactive process for users and bots. By the time they've seen the minimum priority fee has increased for a previous block, the demand may have already dropped. The slow potential for reaction is probably not "punishing" enough for malicious txs that are just trying to eat up blockspace for a short period of time.

I agree that this is an issue, although I don't know that this SIMD addresses that issue.

I think if we were starting Solana today, there's very little chance we'd have a static base fee that was not dependent on the cost of execution for a transaction. The base-fees in this proposal are a replacement for that, not priority markets.

CUs are only one part of the cost of execution; the total number of transactions that can be fit into a block is dependent on many factors, and write locks are a big part of those too. I think it's better to let validators compete on being the best to evaluate the true costs of executing tx relative to other tx instead of encoding some approximation in protocol. Because validators will earn more if they are better at packing blocks, so there is a natural incentive for validators to be good at that, even better than a static rule set for predicting tx execution cost would be. It becomes a differentiating factor for validators; those who earn more block rewards can potentially earn more stake by refunding some of those rewards back to stakers, or reduce commission commensurately, or just be more profitable in general.

[bji]

One of the problems we have seen is that the priority fees do not reliably result in higher priority. In a competitive environment, a low-fee TX will sometimes be processed ahead of another TX with a higher priority fee. Has this been addressed/fixed yet? There might be confounding factors outside the fee structure that should be addressed before we can confidently optimize fees. (People won't use the fees if they don't work).

A leader will begin packing at the beginning of their slot(s). If a low-priority transaction is there, but the high-priority transaction hasn't arrived then low-priority one would probably get packed before it.

Yes tough problem to solve. How much induced latency is profitable? Waiting to initiate execution of tx could risk sub-optimal block packing due to running out of time in the slot. But premature initiation of execution of tx could allow too many "low-paying" tx to push out "high-paying tx".

Validators can compete on being as skilled as possible at making these trade-offs.

Non-conflicting transactions are processed & packed in parallel. So even if both are within the leader's buffer, a low-priority tx could come before a high-priority one if they didn't conflict, but the high-priority was blocked by some other even higher priority tx.

[tao-stones]

Also to remind: this proposal is about base fee -- fee transactions need to pay for being included in block; it is not about priority fee that is paid to increase chances to be considered for block inclusion.

@R-K-H: I suppose I'm curious how this addresses DDoS and Spam with keeping the current stakeholders comfy.

The (exponentially) increased base fee during block space contention should/would provide economic incentive to stop/reduce spamming. On the other hand, the experiment shows such dynamic fee schema would increase validators' income, and lower most transactions' base fee. A win-win is possible.

4. Can add a new instruction to set "max base fee" a transaction willing to pay to simplify UX.

UX can be somewhat improved with this. For instance, wallet can display "transaction fee: <= 0.001 SOL" (if sets max base fee to 0.001 sol, without priority fee)

[bji]

Also to remind: this proposal is about base fee -- fee transactions need to pay for being included in block; it is not about priority fee that is paid to increase chances to be considered for block inclusion.

Good reminder.

I think that base fee should be only and exactly that expected to cover the cost of:

• Receiving tx packets from the network
• Decoding tx

Because that is the only work that a validator is obligated to do to process the tx. A validator cannot even evaluate a tx relative to other tx unless it has done that work.

Once that work is done, priority fees (and evaluation of the expected cost of executing the tx) can be used to determine what tx are included in a block, and are all that are needed for that purpose. A priority fee of 0 is acceptable here because a validator could decide that it has so much extra block space that it doesn't mind processing tx for free.

[apfitzge]

@bji, you are right in this SIMD not having dynamic fees. I was conflating Tao's new comment #19 (comment) with the proposal.

As it stands I don't support the SIMD as is, but am open to the dynamic fee-structure proposed in the comment I've linked.

[iammelea]

Experienced the idea of dynamically setting Compute Unit price based on block utilization. To share some of the findings here:

The idea:

Say there is an ideal block utilization (for example, 81%), a block is said to be at normal block utilization if it's block utilization is within range of ideal block utilization +/- band width. In that case, Compute Unit price shall rest at neutral; If a block's utilization goes above normal block utilization, then block space is under pressure, Compute Unit price shall raise; Or if below normal block utilization, block space is underutilized, Compute Unit price shall fall.

The experiment:

Experimented this idea by replaying mainnet-beta ledger with modified ledger-tool. Have it prints out prioritization fee, signature fee, and vote transaction fee as they are collected today, as well as projected with dynamically determined Compute Unit price.

the setup:

* `block utilization` = bank.cost_tracker.block_cost() / MAX_BLOCK_UNITS

* `ideal block utilization` is determined by running Exponential Moving Average
  of `block utilization` from mainnet-beta ledger, take it's mean value `81%`;

* `normal block utilization` range is 2 stddev of `ideal block utilization`,
  range [74%, 88%] is used in experimenting.

* The initial (or normal) `Compute Unit price` was set to 1 lamport/cu.
  if ema(block utilization) >= 88%, `Compute Unit price` *= 1.125;
  if ema(block utilization) <= 74%, `Compute Unit price` *= 0.875;
  else `Compute Unit price` *= 1.125 or 0.875 until rest at initial value;

* The PoC branch: https://github.com/taozhu-chicago/solana/tree/ledger-tool-experiment-1559-cu-pricing
  Run `solana-ledger-tool verify` built from the branch off of  mainnet-beta
  ledger to print current collected fees, and projected fees.

the limitation

Replaying ledger does not form economic feedback loop. Meaning raised Compute Unit price in experiment does not lower block utilization, as it would in real world. Same for when price is lowered, utilization does not increase in test.

In order to do apple-to-apple comparison, the projected signature fee is calculated as number_of_signatures * 720 * Compute_unit_price, where 720 is a currently defined constant CUs per signature.

In calculating total projected signature fee, a big assumption is made - it is assumed transaction would have to pay 2X (double) signature fee will not land in block (either due to insufficient payer balance, or sender won't send due to higher cost).

the data:

A set of data collected by running experiment with above mentioned setup off of 37,660 mainnet-beta slots, contains 4,389,527 non-vote transactions, proximally 38,040,978 vote transactions.
fees current projected
vote fee 190,204,890,000 same
prioritization fee 15,237,943,001 same
signature fee 23,022,550,000 2,540,877,729

By setting Compute Unit price dynamically based on block utilization, the total collected signature fee reduced by -88.96%. Break it down a bit:

* projected TXs pay more signature fee: 725,395, which is 16.53%

* projected TXs pay less signature fee: 3,664,132, which is 83.47%
  So _most_ users will pay less fee with dynamic pricing;

For validators:

* with current 50% burn of all fees collected, validators collect: 114,232,691,501

* if 100% base fee burned, and 100% prioritization and vote fee go to validators,
  validators are projected to collect: 205,442,833,001
  which is a 79.85% increase in fees collected, also a win for validators

So, who are the losers?

Those 16.53% TXs that paying more signature fee are paying a lot more. In this experiment, Compute Unit price increases exponentially, without cap. So block utilization stay above normal block utilization continuously for several slots, the Compute Unit price can go sky high. This is a limit of replaying historical data.

Additional thoughts:

1. Using signature fee above is for comparison sake, transaction base fee should
   be `= transaction.total_cost() * Compute_unit_price`, where `total_cost()`
   is the same number used by block producer to pack blocks, so to archive the
   symmetry in block packing and fee charging.
   This will likely significantly increase cost for transactions that request
   large cu_limit.

2. Prioritization fee should be expected to increase, perhaps significantly,
   during high block utilization period. This would in turn impact both users
   and validators economics.

3. UX. Additional tools are likely necessary to improve UX with dynamic pricing.

4. Can add a new instruction to set "max base fee" a transaction willing to pay to simplify UX.

this is the best idea for decentralized the solana chain and attract more validators to the validator set, thanks for this idea

[bji]

Interesting data, but please re-compute with 50% base fee burned. Base fee needs to go to validators, it pays the one unavoidable cost that validators must pay to process tx, which is cost to receive and decode tx.

50% burn is OK, but it can't be 100% burn.

50% priority fee burn should probably be the case also, because otherwise validators can pay themselves very high priority fees to mess up any mechanisms that are based on tracking avg priority fee over time.

[tao-stones]

Updated above comment with following scenario:

- if 50% base fee burned and prioritization fee are burned, the rest and 100% vote
  fee go to validators, validators are projected to collect: 199,094,300,365, which
  is a 74.3% increase in fees collected.

[apfitzge]

Base fee needs to go to validators, it pays the one unavoidable cost that validators must pay to process tx, which is cost to receive and decode tx.

Not convinced by this argument. Validators don't get base-fee rewards because they received and decoded the tx, they get it for packing it in their block.

50% burn is OK, but it can't be 100% burn.

The proposed mechanism for base-fee is exponentially increases and decreasing. If the leader receives any of those rewards, they are incentivizd to keep the base-fee high.
This could potentially reward behavior such as sending spam transactions to the leader preceding your leader slots - that way the base-fee is higher for your blocks.

We could probably do some math to find what percent makes this sort of strategy unprofitable. However, I think considering there's usually some base-level of load that's reasonably close to the threshold (remember the fee changes are determined by divergences 2stddev from mean load) it's likely that you wouldn't need too many transactions to bump it out of this range.

---

Example:
Node 1 is current leader.
Node 2 is leader in 4 slots.
Current base-fee is 10 lamports/cu.
base-load is mean of 81%.

To get above 88% threshold for raising prices, Node 2 will send 3.36M CUs of spam for next 4 slots.
Slot 1: 33.6M lamports (@ 10 l/cu)
Slot 2: 37.8M lamports (@11.25 l/cu)
Slot 3: 42.5M lamports (@12.65 l/cu)
Slot 4: 47.8M lamports (@14.24 l/cu)
Total: 161.7M lamports

The price is now 16.02 l/cu for my slots. Assuming mean load of 81%: 2491.4M lamports (4 slots @16.02 l/cu) - w/ 50% burn this would be 1245.7M lamports reward.
If node 2 didn't spam, this would have been 1555.2M lamports fees w/ 777.6M in rewards.

Node 2 would effectively 3x their "investment" on those spam txs.

In this example, the starting price was already high. But it shows that in these congestive periods this sort of strategy is incentivized specifically at the time we don't want it to be.

[tao-stones]

My intuition agrees @apfitzge's argument - no one should benefit from global resource congestion, but everyone should pay to contribute to such congestion. so protocol to burn 100% dynamic base fee makes sense to me.

[R-K-H]

My intuition agrees @apfitzge's argument - no one should benefit from global resource congestion, but everyone should pay to contribute to such congestion. so protocol to burn 100% dynamic base fee makes sense to me.

I agree, however I do think @bji has a point, regardless of the behavior, there is a cost incurred on the validators side via bandwidth and cpu cycles. Is this accounted for within the model?

As well doesn't the same argument apply to priority fees or is that localized to the "hot" accounts?

[apfitzge]

there is a cost incurred on the validators side via bandwidth and cpu cycles. Is this accounted for within the model?

The use of bandwidth and compute is requried for participation by all nodes, not just the leader.
Any leader-only rewards imo, would not be sufficient for what you're asking if I understand correctly.
The leader is rewarded for the bandwidth and compute usage via priority fees they collect on the transactions they pack.

Potentially there's some economic changes to epoch-level rewards where all nodes are rewarded for base-level participation - this seems like an entirely different proposal and discussion beyond this congestion control.

[brianlong]

A leader will begin packing at the beginning of their slot(s). If a low-priority transaction is there, but the high-priority transaction hasn't arrived then low-priority one would probably get packed before it.

This reinforces the point I was making. People won't use PF if they work only some of the time.

[brianlong]

Also to remind: this proposal is about base fee -- fee transactions need to pay for being included in block; it is not about priority fee that is paid to increase chances to be considered for block inclusion.

Thanks. This fixes my concern about PF not working reliably. Are we still left with a situation where simple transfers become expensive?

[brianlong]

Don't forget the cost of storing the TX/ledger in long-term storage. Currently, we do not collect fees for long-term storage. The price to write a TX should include the expense to save it forever in the ledger. I expect to rough out another SIMD for storage costs at Breakpoint this year. Teaser: Show up for Block 0 (Validator Day)!

[brycole]

Don't forget the cost of storing the TX/ledger in long-term storage. Currently, we do not collect fees for long-term storage. The price to write a TX should include the expense to save it forever in the ledger. I expect to rough out another SIMD for storage costs at Breakpoint this year. Teaser: Show up for Block 0 (Validator Day)!

This is a legit issue. People are already talking about the ledger to store nfts similar to ordinals. If it's not an issue now it will be sooner then we think.

[tao-stones]

Are we still left with a situation where simple transfers become expensive?

when block space under stress (eg utilization > 88%), all transactions including simple transfer will become more expensive, otherwise one should expect simple transfers become cheaper.

[brianlong]

when block space under stress (eg utilization > 88%), all transactions including simple transfer will become more expensive, otherwise one should expect simple transfers become cheaper.

If the fees are too damn high, the blocks are too damn small!

Thanks for answering my questions. I'll re-read this proposal over the weekend.

[ripatel-fd]

@anoushk1234 @jacobcreech While I appreciate the effort that went into this proposal, and is worth discussing, I suggest closing this PR or setting it to draft stage.

As it stands, this proposal presents a list of suggested changes, but it fails at specifying what these changes exactly are. It also doesn't clearly demonstrate how the suggested fix solves the problem mentioned. (Most of this is in various comments on this PR).

In my humble opinion, if a SIMD cannot be unambiguously implemented, the idea should be further discussed via Discord/Forum, etc, before it enters the improvement process.

[jacobcreech]

While I appreciate the effort that went into this proposal, and is worth discussing, I suggest closing this PR or setting it to draft stage.

@anoushk1234 @ripatel-fd I agree.

As per SIMD-1, let's continue the discussion on this topic within #core-technology on the Solana Tech Discord. Once a design is met and fleshed out that has a chance of reaching consensus, feel free to open another SIMD and we can take it through review again.