Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use lazily loaded Merkle Tree root as EC Chain Key #825

Closed
masih opened this issue Jan 15, 2025 · 0 comments · Fixed by #834
Closed

Use lazily loaded Merkle Tree root as EC Chain Key #825

masih opened this issue Jan 15, 2025 · 0 comments · Fixed by #834
Assignees
@masih
Milestone
M2: Mainnet Passi...

Comments

@masih
Copy link
Member

masih commented Jan 15, 2025

The current implementation uses two approaches for generating key for a given EC Chain:

  1. ChainKey construction by hashing each of the tipsets
  2. Merkle Tree root hash used in signing the messages.

There are historical reasons for this: e.g. the first approach predates the second. But there is no technical reason for having two separate approaches.

In terms of computational expense, the first approach is cheaper than the second one. But in practice:

  • we have to calculate the root Merkle tree hash once for signing purposes.
  • the code repeatedly generates chain key using the first approach.

This means, if merkle tree root is generated once and cached as part of ECChain struct the computational expense becomes a non-issue if not cheaper.

Additionally, this approach would benefit #792, because chainexchange no longer needs to own the concept of generating keys for chains. We can just call ECChain.Key and use the same key across the entire implementation.

On approach, the idea is to use sync.Once and cache the merkle tree root as key lazily.
@github-project-automation github-project-automation bot moved this to Todo in F3 Jan 15, 2025
@masih masih self-assigned this Jan 15, 2025
@BigLep BigLep added this to the M2: Mainnet Passive Testing milestone Jan 16, 2025
@BigLep BigLep moved this from Todo to In progress in F3 Jan 16, 2025
masih added a commit that referenced this issue Jan 20, 2025
@masih
Consistently use Merkle Tree root as ECChain key
2065e1f 
Remove the custom ECChain key generation inside `gpbft` in favor of
consistently using Merkle Tree root of chain as key. This key is already
used for signature payload generation, and in the new chain exchange as
a key to identify a chain. Doing so enables signature validation of
messages without having to know the chain.

Consistent use of Merkle Tree root hash as key enables a number of
simplifications across the repo:
* unblocks the unification of validation logic between partial and full
  validators, since caching can use a consistent way to identify
  messages whether they're partial or not.
* reduce various type gymnastics across root and chain exchange packages
  by repurposing gpbft ECChain Key.

The work here also introduces lazy-loading for the ECChain, which
requires ECChain type to be converted to `struct`, and be passed by
pointer. As part of this work, TipSet is also turned into a pointer
consistently across various interfaces.

The ECChain receiver functions are adopted to accommodate potential nil
values for a chain.

Fixes #825
@masih masih mentioned this issue Jan 20, 2025
Merged
@BigLep BigLep moved this from In progress to In review in F3 Jan 20, 2025
masih added a commit that referenced this issue Jan 20, 2025
@masih
Consistently use Merkle Tree root as ECChain key
ce61a37 
Remove the custom ECChain key generation inside `gpbft` in favor of
consistently using Merkle Tree root of chain as key. This key is already
used for signature payload generation, and in the new chain exchange as
a key to identify a chain. Doing so enables signature validation of
messages without having to know the chain.

Consistent use of Merkle Tree root hash as key enables a number of
simplifications across the repo:
* unblocks the unification of validation logic between partial and full
  validators, since caching can use a consistent way to identify
  messages whether they're partial or not.
* reduce various type gymnastics across root and chain exchange packages
  by repurposing gpbft ECChain Key.

The work here also introduces lazy-loading for the ECChain, which
requires ECChain type to be converted to `struct`, and be passed by
pointer. As part of this work, TipSet is also turned into a pointer
consistently across various interfaces.

The ECChain receiver functions are adopted to accommodate potential nil
values for a chain.

Fixes #825
masih added a commit that referenced this issue Jan 20, 2025
@masih
Consistently use Merkle Tree root as ECChain key
b569f7f 
Remove the custom ECChain key generation inside `gpbft` in favor of
consistently using Merkle Tree root of chain as key. This key is already
used for signature payload generation, and in the new chain exchange as
a key to identify a chain. Doing so enables signature validation of
messages without having to know the chain.

Consistent use of Merkle Tree root hash as key enables a number of
simplifications across the repo:
* unblocks the unification of validation logic between partial and full
  validators, since caching can use a consistent way to identify
  messages whether they're partial or not.
* reduce various type gymnastics across root and chain exchange packages
  by repurposing gpbft ECChain Key.

The work here also introduces lazy-loading for the ECChain, which
requires ECChain type to be converted to `struct`, and be passed by
pointer. As part of this work, TipSet is also turned into a pointer
consistently across various interfaces.

The ECChain receiver functions are adopted to accommodate potential nil
values for a chain.

Fixes #825
masih added a commit that referenced this issue Jan 24, 2025
@masih
Consistently use Merkle Tree root as ECChain key
d15b409 
Remove the custom ECChain key generation inside `gpbft` in favor of
consistently using Merkle Tree root of chain as key. This key is already
used for signature payload generation, and in the new chain exchange as
a key to identify a chain. Doing so enables signature validation of
messages without having to know the chain.

Consistent use of Merkle Tree root hash as key enables a number of
simplifications across the repo:
* unblocks the unification of validation logic between partial and full
  validators, since caching can use a consistent way to identify
  messages whether they're partial or not.
* reduce various type gymnastics across root and chain exchange packages
  by repurposing gpbft ECChain Key.

The work here also introduces lazy-loading for the ECChain, which
requires ECChain type to be converted to `struct`, and be passed by
pointer. As part of this work, TipSet is also turned into a pointer
consistently across various interfaces.

The ECChain receiver functions are adopted to accommodate potential nil
values for a chain.

Fixes #825
masih added a commit that referenced this issue Jan 24, 2025
@masih
Consistently use Merkle Tree root as ECChain key
eb6df84 
Remove the custom ECChain key generation inside `gpbft` in favor of
consistently using Merkle Tree root of chain as key. This key is already
used for signature payload generation, and in the new chain exchange as
a key to identify a chain. Doing so enables signature validation of
messages without having to know the chain.

Consistent use of Merkle Tree root hash as key enables a number of
simplifications across the repo:
* unblocks the unification of validation logic between partial and full
  validators, since caching can use a consistent way to identify
  messages whether they're partial or not.
* reduce various type gymnastics across root and chain exchange packages
  by repurposing gpbft ECChain Key.

The work here also introduces lazy-loading for the ECChain, which
requires ECChain type to be converted to `struct`, and be passed by
pointer. As part of this work, TipSet is also turned into a pointer
consistently across various interfaces.

The ECChain receiver functions are adopted to accommodate potential nil
values for a chain.

Fixes #825
github-merge-queue bot pushed a commit that referenced this issue Jan 24, 2025
@masih
Consistently use Merkle Tree root as ECChain key (#834)
ded3d04 
* Consistently use Merkle Tree root as ECChain key

Remove the custom ECChain key generation inside `gpbft` in favor of
consistently using Merkle Tree root of chain as key. This key is already
used for signature payload generation, and in the new chain exchange as
a key to identify a chain. Doing so enables signature validation of
messages without having to know the chain.

Consistent use of Merkle Tree root hash as key enables a number of
simplifications across the repo:
* unblocks the unification of validation logic between partial and full
  validators, since caching can use a consistent way to identify
  messages whether they're partial or not.
* reduce various type gymnastics across root and chain exchange packages
  by repurposing gpbft ECChain Key.

The work here also introduces lazy-loading for the ECChain, which
requires ECChain type to be converted to `struct`, and be passed by
pointer. As part of this work, TipSet is also turned into a pointer
consistently across various interfaces.

The ECChain receiver functions are adopted to accommodate potential nil
values for a chain.

Fixes #825

* Bump timeout for `TestF3LateBootstrap` to unblock progress

* Bump timeout for `TestF3PauseResumeCatchup` to unblock progress

* Bump timeout for `TestF3LateBootstrap` to unblock progress

Bumped timeout to 2 m, just to see if CI passes.
@github-project-automation github-project-automation bot moved this from In review to Done in F3 Jan 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@masih masih

Labels
None yet
Projects
F3
Status: Done
Milestone
M2: Mainnet Passive Testing
Development

Successfully merging a pull request may close this issue.

Consistently use Merkle Tree root as ECChain key filecoin-project/go-f3
2 participants
@BigLep @masih