Problems with storage staking

[akhi3030]

The world as is it today

Storage staking

Today contracts need to stake NEAR tokens to pay for the storage that they are using. The more state an account is using, the more tokens it needs to hold. The actual payment is the opportunity cost of not being able to stake the tokens that are now locked up to support the storage that the contract is using. The basic idea is that the opportunity cost would create an incentive for the contract owners to remove state that does not need to exist on the chain anymore. And this incentive would help in ensuring that users will remove garbage state from the chain.

Zero balance account

An additional factoid to keep in mind is that since the zero balance account NEP was stabilised, for up to 770 bytes of storage, accounts do not have to stake any tokens. The payment for this storage happens when the account is created (in the form of the account creation fee) and then the account has this much storage available to use in perpetuity.

The potential problem that this has created is that there is no motivation to cleanup the storage (up to 770 bytes) that an account is using because removing that state or deleting that account does not actually allow the user to recover any tokens. In general though, we are not as worried about this problem because the thinking has been that the number of accounts needed to create a large enough amount of garbage state to truly impact the protocol is going to be quite large.

Where are we moving to?

There is a new NEP currently being discussed that is going to add some twists to above. This NEP is effectively proposing that for a one time fee, I can buy an arbitrary amount of storage in perpetuity.

The problem

The problem arises because of one of the use cases that wants to use this feature. The use case is that a third party wants to onboard users to NEAR; so they create accounts for them; and then give control of the accounts to the users. This is all well and fine. However, imagine if after trying out the service, 50% of the users lose interest in it and decide to stop using it or if the third party goes bankrupt and stops supporting their users. The users do not have any incentive to delete their state because as proposed by the NEP, they will not stand to gain any refunds from doing so. So chances are that the users will simply walk away and there will be garbage state left on the protocol which it will have to maintain in perpetuity.

One could argue that the third party could create an incentive for the users to clean up their state before they walk away. However, I see one of two things happening here:

• The incentive per user will be around the number of NEAR tokens for a single account and might be worth a few dollars at most. So it might not be enough incentive for the users to actually go through with the process (e.g. how do they collect the refund; do they have to do KYC; provide bank details; etc.)
• If the third party creates an incentive to delete the accounts, then they are also creating an attack vector. As seen in this comment, this incentive can lead to malicious users creating and deleting accounts to try to steal the small amount of tokens per account.

Problem summary

To summarise, currently there is a problem on the protocol: there is no incentive for users to clean up the storage that their accounts are using up to 770 bytes leading to accumulation of garbage state on the protocol. The understanding is that this will probably not lead to a lot of garbage state on the protocol though. And if the proposed NEP is accepted, then chances of garbage state accumulating on the protocol is further increased.

Alternative approach

If we look at related works, there is a common approach adopted by other systems. Instead of selling storage; the protocol could rent it. This is how almost all cloud storage services and many other blockchains also work. Users of storage need to keep paying on regular intervals for the storage that they are using and if they ever stop paying for it, eventually the storage will be deleted.

In this mechanism, someone is actively interested in keeping state on the chain and will continue to pay for it and if no one is interested in keeping the state on the chain, then eventually it will be deleted when the rental payments stop.

This post is already quite long so I will not go into further details on how such a rental system could look like. But happy to expand if there is interest in pursuing this approach.

[birchmd]

This NEP is effectively proposing that for a one time fee, I can buy an arbitrary amount of storage in perpetuity.

I don't think this is a charitable interpretation of the NEP. The proposal does not fundamentally change the way storage staking works, it just creates an alternate form of staking that is not trivially susceptible to spam. The sponsored accounts still have bounded storage depending on how much NEAR has been given to it.

That said, I agree the overall argument that an inevitable consequence of this NEP is more "garbage state" on the network. This is funny in a way because it means that the opportunity for spam protecting the protocol is in a sense.

I think what this discussion highlights is that the storage staking model puts a cost on storage which is not directly connected to the underlying cost of maintaining that storage. If the storage is more valuable to the account holder than the staking opportunity cost then that entity has an incentive to create that storage; garbage or not. In the context of the proposed NEP, the company will continue to sponsor accounts (even if many end up unused in the long run) if the expected lifetime value of a user exceeds the cost of acquiring the user.

But these factors (lifetime value and acquisition cost of user to a dapp) are only indirectly connected to the validators who have to pay for the physical hardware that this storage is persisted on. Increased usage of the platform could increase the Near token price, which would be good for validators, but it is also bad for the dapp (their user acquisition cost goes up). Overall, the economics looks complicated to me and I don't feel qualified to say whether or not the storage staking model "should" work.

A storage rent model might match the cost of the validators more closely (cloud storage tends to be rented as you mentioned). If the cost of the physical disks is always met by those that created the storage usage then validators should not have any issue with the state growing (whether that state is "garbage" or not). Though it is not clear to me how we could guarantee this property holds if storage is charged in the Near base currency (since then the storage cost would fluctuate with the Near price). It's also not clear to me if this cost needs to scale with the number of validators (since the storage is replicated on each validator machine). If so, that is additional complexity.

tl;dr I don't think the proposed NEP breaks the storage staking model. It makes storage staking easier to use for a certain use-case and maybe storage staking was always broken for that use-case (or maybe it's not). Storage renting is an interesting idea and if you have more details on how it would work then I would love to hear them.

[dndll]

I think storage rental is an excellent idea, could the initial staked funds could be metered, and the protocol could remove some of it after some time, or the funds reach 0, or less? However, it was raised before that this would be an additional UX process for the user, and they would see their staked funds run down in perpetuity in the wallet, which might need to be clarified for existing users.

I also agree with @birchmd that it might only partially represent the exact cost to validators to the dollar, but a mechanism to provide something in the right direction cost in NEAR seems like a good idea.

I'm interested in additional approaches to a rental model and how we could have both a predictable model and something that limits user complexity.

From a technical perspective, it makes sense that we should try to incentivise deleting "garbage state" from the network.

It might even be interesting for developers to see what state is "garbage" and how to clear it. Perhaps some developers don't even realise that particular storage entry hasn't been used in some period of time and how they could remove it, potentially incentivising the users that report it with some of the released funds. This seems like a use case that would have to be implemented on the contract layer but would encourage some markets where fishermen-like people look for a "garbage" state, try to get it cleared and get reimbursed for their effort.

[jakmeier]

Thanks for writing this down!

I see the fundamental problem and I agree that storage rent is a possibly nice solution for it. However, firstly, I don't see how this is in any conflict with NEP-491. Second, I'm have practical concerns whether storage rent can really solve the issue.

1. How is it related to NEP-491?

A basic storage rent implementation doesn't solve the storage refund attacks, so it's not an alternative to NEP-491, right? Nor would it be impossible to implement storage rent after we ship NEP-491 as proposed. Like, the refundable amount that a sponsor pays could just go into a budget that's locked for paying storage rent only.

So from my point of view, this is a great proposal which we should discuss and see if we can come up with a complete design. But I don't see it as a blocker for NEP-491, which tries to resolve a different pain point in a more immediate time frame.

Does that make sense? Please let me know if I misunderstand something or just generally miss the point of the proposal here.

2. Does storage rent solve the problem?

For the rent vs staking costs, I think it's important to look at actual numbers. I've compiled a table comparing the costs in NEAR with cloud providers and other decentralized storage solutions some time ago: near/nearcore#8492 (comment)

You will see that storing something on S3 for 100s of years at current prices is still cheaper than the one-time-payment in NEAR. And yet, smart contract maintainers find it to be "not worth it" to reclaim the storage-staked tokens of their inactive users. It's just too few bytes and acquiring a user in the first place is many orders of magnitudes more expesnive.

Which makes me think: Even if we come up with a storage rent solution, the current behavior of developers maps to the storage rent model such that they will just pre-pay enough tokens that they never need to worry about paying rent again. For example, fund 100 years of storage rent for all users. Which leaves us with the same amount of garbage state for all practical purposes.

3. What would I suggest?

I think we need a completely different solution. Ideally, we have a way to make all useless data cheaper to store, even if there was something prepaid. (Cheaper for validators, that is.)

As I like to say, financial incentives don't work to solve fundamentally technical problems. In this case, the $$$ numbers just don't work in our favor. Making the price high enough that it isn't "abused" by sponsors to fill the chain with garbage would price-out most private individuals. And that's true regardless of how we charge it, as rental model, one-time payment, or as staking fee.

But I think we can achieve a technical solution. For example, we can offload trie state that wasn't used in 1 year to a different storage solution, like IPFS. Maybe we can use the treasury to pay for data pinning services, or maybe we say that's the responsibility of developers and end-users.

Then if a user tries to access inactive data, the protocol automatically and smoothly reloads it into the active trie. We can charge extra gas or tokens in this case for all the overhead. But more importantly, we would delay the execution by a couple minutes to fetch the data from somewhere.

Of course, this needs a lot of work across the stack. It would require a major commitment to actually implement it. Thus, my main concern is that it will never happen simply because the pain of garbage data isn't felt enough. And maybe that's a sign that we are speculating too much about problems that aren't even that big, yet. Whereas the refund problem is real and prevents many use cases on Near Protocol right now.

[akhi3030]

Thank you very much for your feedback and thoughts. I am working on a more detailed post and I will write it shortly. In the meantime I just wanted to address one of @jakmeier's comments. I do not think that this proposal is an alternative to NEP-491 and I am not proposing it to block NEP-491. I think we should continue with NEP-491 because it does address an important issue that we have today. I am trying to propose a more longer term solution.

[Ekleog-NEAR]

Correct me if I’m misunderstanding, but I do think that a storage rent model would actually remove the need for NEP-491.

The issue NEP-491 solves is basically the storage refund problem, whereby a service stakes to create an account for an user, who then deletes it in order to get the refunded stake.

With a storage rent model, the service would not stake to create an account for an user, but would pay the rent for this account. As such, the user could never get back any refunded stake, because there was never any in the first place. And when the service wants to stop paying for the user’s rent, the charge of paying for it will just fall back on the user themselves.

As for the costs themselves, I think it would make sense to set them, such that the current staking opportunity cost is the same as the rent we expect from users.

So overall, I do think that this proposal would remove the need for NEP-491. However, it might be too long to implement, to be able to do that in time for the planned use cases of NEP-491, I don’t have any timeline for these in mind.

---

This being said, I’m completely in line with @jakmeier that IMO, neither of these ideas is a good solution for the problem of cruft accumulating on-chain. And that we definitely need to do something to reduce the cost for both validators and end-users, because the table Jakob linked is definitely a problem for everyone.

(That said, we still need to take into account archival nodes costs if we want to keep them, and these won’t benefit from any state removal opportunity, quite the contrary… but they can "just" be paid by the treasury I guess)

[jakmeier]

With a storage rent model, the service would not stake to create an account for an user, but would pay the rent for this account.

If you assume the option exists to pay the rent for someone else (without just sending them the money), then yes, it makes NEP-491 obsolete. But that sounds like two features to me.

I assumed basic storage rent is taken from the owning account, just like in the original implementation before the mainnet launch. If you want to solve the refund problem, you have to come up with something on top, like paying rent for someone else. But equally we could allow to stake for someone else, so I don't think this solution is unique to the storage rent model.

[Ekleog-NEAR]

I assumed basic storage rent is taken from the owning account

Right, I assumed storage rent could be paid by a third party. Staking for someone else seems significantly harder, because you need to remember how much to unstake for each part of the state (as otherwise the user would have to "buy back" their account from the service provider for a significant amount even for 1B of overdraft, which is bad UX), as opposed to just remembering from which contract to take the rent.

That said, even without a "rent paid by third party" model, a storage rent model would already be a better situation regarding theft than our current model, though it’d be less good than NEP-491. Because currently the user could steal the whole staked amount, whereas with an eg. epoch-based storage rent model the user could only steal roughly 10% / 730 ~= 0.01% of the amount, and stealing it would automatically delete the account. It might be enough make the attack not economically feasible, depending on how costly account creation is.

The drawback would be, the cost for the service provider to send that money to each account on each epoch, and some load on the network. But then it becomes a risk-cost decision for the service provider, who can decide on how hard to make the captcha for account creation based on it. And arguably such simple but transaction-number-intensive load on the network would probably be a good thing for marketing 😅