HIPE for plugin helpers

[esplinr]

Introduces features previously discussed in the Indy Contributors calls: frozen ledgers and default fee handlers.

[askolesov]

Looks wonderful. Just a couple of minor suggestions.

[askolesov]

Cool. I think the document is ready. Just a couple of optional suggestions from my side.

[esplinr]

@askolesov : I'm concerned that I was misunderstanding when I explained this drawback:

If auth_rules with fees are defined, but no plugin is installed to process them, the validation will fall back to the default fee handler which will ignore them and consequently grant authorization. This permissive behavior could be dangerous in some circumstances, and network administrators should ensure that auth_rules do not rely on fees when no plugin is installed to handle them. This is consistent with other Indy configuration options which are permissive by default to assist new users, but expected to be locked down for production use.

Since we also say:

The default fee handler will contain a copy of the sovtoken fee handler but with validation disabled, so it will allow catchup but will not allow new transactions with fees.

Doesn't that mean that the behavior is not permissive? Meaning all auth_rules that exclusively rely on fees for authorization will always fail unless their is a token plugin to override the default fee handler?

I think I need to rewrite the paragraph about the drawback.

[askolesov]

@esplinr It's important to distinguish two cases:

1. Somone tries to execute a transaction that has a fee defined. There will be no logic to decrease the author's balance without plugin so the information that the transaction has fee will be simply ignored. And this is the permissive behaviour you are talking in the first part.
2. Someone tries to change fee for a transaction (i.e. sends SET_FEE transaction). This is what your second cite is about. In this case, the transaction will be rejected. So yes, it's not permissive behaviour.

[m00sey]

Reading this I have an overarching feeling it's two HIPEs

one for ledger freezing, a QoL improvement for ledger plugin/development
and the second for the the introduction of fees at an Indy (vs plugin) level

I appreciate both are needed to resolve the current sovtoken issues, but feel they are worth discussing independently as I think they individually provide value

[m00sey]

I understand the sentiment but I don't think it reflects the true origin of this HIPE.

There was every intention for the token ledger to be used initially. Given the nature of ledger plugins, freezing is a "Quality of life" improvement allowing ledgers to be implemented and replaced over time? In this case the token ledger never reached fruition, but future ledgers may become defunct or replaced by a "better version" and I think the language should reflect that.

"If a ledger has been created, but has become defunct or redundant, it can be frozen"

[esplinr]

This touches on two separate properties of our blockchain:

• The ability to achieve consensus when writing a new transaction to the ledger.
• The ability to audit the history of the ledger to prove that there was no tampering.

When a ledger is frozen, the root hash of the deprecated ledger is available to be used by the audit ledger when computing consensus. But we do not store the entire history of root hashes, so it does not help if someone tries to recompute a history that relies on the removed ledger. The proposed implementation of frozen ledgers only works for Sovrin because MainNet's token ledger was unused, and we don't guarantee historical reliability on StagingNet and BuilderNet.

I expanded on the explanation in the "Drawbacks" section to hopefully make that more clear.

[m00sey]

I'll take a look, thank you.

[m00sey]

I 100% agree with this to me it's a feature of ledger plugins and belongs in Indy

[esplinr]

For reviewers: a convenient link to the pull request for the Sovrin SIP that is linked in the HIPE.

[esplinr]

PR's that implement the functionality described here:
hyperledger/indy-plenum#1502
hyperledger/indy-node#1641
hyperledger-archives/indy-sdk#2356

[lynnbendixsen]

This HIPE solves the issue in an acceptable way.

[WadeBarnes]

Using the Sovrin Ledgers as an example, the token ledger has been used on BuilderNet and StagingNet, but not on MainNet. Based on this statement it would be safe to delete the token ledger data from MainNet, but not on BuilderNet, or StagingNet. If the ledger were to be deleted from either BuilderNet, or StagingNet the ability to audit the history of the ledger would fail, but how would that present itself, and how do we protect against deleting a ledger that has been used so we can ensure the historical integrity of the ledgers?

[WadeBarnes]

Added an issue against the remove_ledger.py script; hyperledger/indy-node#1659

[esplinr]

HIPE 0162 addresses this in the Drawbacks section.

3. The ability to audit the history of the ledger to prove that there was no tampering.
   … the third property will not be preserved if there is a history of transactions on a ledger and then the ledger is removed. This is because the LEDGERS_FREEZE transaction only stores the most recent root hash, and not the entire history of root hashes, so that history can not be recreated after the plugin ledger is removed. Instead, the frozen ledger should be retained in its read-only state.

As the HIPE explains at the beginning,

The data stored on the custom ledger created by the plugin will be deleted.
[This] consequence is intended.

The associated SIP makes clear that this is acceptable for the Sovrin test networks because they are not intended to have a permanent history. See my comment on the related sovrin-sip.

[WadeBarnes]

The answers focus more heavily on the current intended use case. I'm thinking more about the impact when something like this is used in general and it's impact to historical integrity.

To summarize my understanding so far:

Once the the ledger has been frozen and deleted, the ability to audit the history is irrevocably lost.

• How does this end up presenting itself from an operational and network administrative standpoint?
• What are the implications to the network and other ledgers?
• What is the impact to the network?

If the ledger is frozen but the data is retained, can the history of the ledger be audited?

• Can this be accomplished if the plugin that created/managed the ledger is removed?

[esplinr]

I don't really understand your questions. Perhaps the confusion is due to different uses of the word "audit".

The audit ledger exists to confirm the relationship between all of the ledgers if someone decides to replay the transaction history. But this isn't something that the ledger does. It is something that a third party could theoretically do if desired.

How does this end up presenting itself from an operational and network administrative standpoint?

As described in the "Drawbacks" section, the network continues to function correctly. New transactions can be ordered. Catch-up continues to function, because the validation logic is not executed.

But attempting to reconstruct the ledger by replaying all events will fail because some of the data is missing. Freeze_Ledgers only stored the root hash of the last entry on the deleted ledgers, so you can't calculate the previous root hashes when checking for consistency in the audit table.

What are the implications to the network and other ledgers?

Things work fine, but third parties who try and recreate the ledger can't have confidence that the history wasn't tampered with (because it was tampered with).

What is the impact to the network?

It works fine. Consensus is preserved, new transactions can be ordered, and queries are processed.

If the ledger is frozen but the data is retained, can the history of the ledger be audited?

Sure. All the data is there.

Can this be accomplished if the plugin that created/managed the ledger is removed?

Yes, but it would take some work to figure out what the removed plugin was doing. This is explained in more detail in the HIPE for default fee handlers: "The default fee handler replays transactions without performing validation. This is fine for catching up validator nodes, but an audit of the ledger history would need to take into account the historical validation expected by any removed plugins."

Exactly how to audit a ledger seems to be beyond the scope of this HIPE.

[WadeBarnes]

Would this then be a fair summary?

Once a ledger is frozen and the associated plug-in is removed, regardless of whether or not you keep the data, the history of the associated ledger(s) cannot be audited. In order to audit their history you would have to retain the ledger data (not delete the ledger after freezing it) and restore the plugin (or computational equivalent) first.

[WadeBarnes]

How does a frozen ledger affect new nodes added to the network. I assume, whether or not the frozen ledgers data was retained, only the hash at the point in time the ledger was frozen will be replicated to any new nodes. Is this correct? I think this may already be implied by the fact the frozen ledger will not be included in catch up operations.

[esplinr]

Would this then be a fair summary?
Once a ledger is frozen and the associated plug-in is removed, regardless of whether or not you keep the data, the history of the associated ledger(s) cannot be audited. In order to audit their history you would have to retain the ledger data (not delete the ledger after freezing it) and restore the plugin (or computational equivalent) first.

I disagree with the sentiment that the history cannot be audited even if you keep the data. The audit process is the same as before, because audits are done outside the ledger software and independent of what plugins are installed on the ledger. To perform an accurate audit you will need to track the history of the software that was installed so that you can confirm the validation rules that were used for the original ledger writes.

[esplinr]

How does a frozen ledger affect new nodes added to the network. I assume, whether or not the frozen ledgers data was retained, only the hash at the point in time the ledger was frozen will be replicated to any new nodes. Is this correct? I think this may already be implied by the fact the frozen ledger will not be included in catch up operations.

I believe that the ledger data is propagated during catch up to the rest of the pool, even for frozen ledgers. This is because validation is not done for transactions that already have a BLS signature. Is that correct @Toktar ?

[esplinr]

I see this note from @anton.denishchenko: "Frozen ledgers are not catched up (this is possible because the LEDGERS_FREEZE transaction is part of the config ledger, which is applied before additional ledgers)."
Thank you @WadeBarnes for highlighting this misunderstanding. I'll add a note about this topic to the HIPE.

[WadeBarnes]

We'll continue the conversation from here (hyperledger/indy-node#1659 (comment)) on this PR.

[esplinr]

I believe I have incorporated all the feedback and answered all of the questions.

[WadeBarnes]

and new nodes in the pool should manually backup the data from frozen ledgers since they are not included in catchup.

This would be additional administrative overhead and could easily be overlooked or done incorrectly. It's definitely something that needs to be addressed at some point before too long. I would think It would be best addressed by the existing catch-up process.

How much effort would be required to support catch-up of frozen ledgers? The need for this should at least be captured in a ticket so it's not forgotten.

[esplinr]

It's too much work for that feature without someone planning to use it, but here is the issue:
hyperledger/indy-plenum#1524