Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FIP-0090: Non Interactive PoRep #891

Merged
merged 45 commits into from
Mar 16, 2024
Merged

Conversation

lucaniz
Copy link
Contributor

@lucaniz lucaniz commented Dec 18, 2023

This proposal presents a new PoRep protocol (Non-Interactive PoRep) that removes PreCommit when onboarding CC sectors. As a result, we have:

  • Simplified onboarding pipeline, unblocking SupraSeal's full potential
  • Trustless separation between storage and computing: the proving tasks associated with sector onboarding can be outsourced.

Discussion: #854

(PR description edited by @anorth)

FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
A first step to mitigate the downsides of the waiting time was the introduction of Synthetic PoRep (See [FIP-0059](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0059.md)), which reduces the size of the temporary data stored between PreCommit and ProveCommit.
NI-PoRep is a further step forward, completely foregoing on-chain interaction (ie, the waiting time) and the need of PCD by allowing SP to locally generate challenges instead of using on-chain randomness.

NI-PoRep has little downside with respect to the status quo: it removes PreCommit at the cost of augmenting C2 (ie SNARK generation) costs, which would result in a limited cost increase looking at onboarding costs as a whole). Indeed, NI-PoRep requires 12.8x more PoRep Challenges, which translates into an 12.8x SNARK proving overhead. We analyzed how this SNARK computation overhead affects overall costs. The conclusion is that considering PC1+PC2+C1+C2 and storage costs (i.e. not considering maintenance costs), a NI-PoRep sector with 128 bits of security is 5% more expensive than an Interactive PoRep sector when sector duration is 3y. See full analysis [here](../resources/fip-xxx-niporep/NIPoRep_CostAnalysis.pdf).
Copy link
Member

@jennijuju jennijuju Mar 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Peer(can move to discussion, doesn't block draft): Why is the worst-case scenario for security modeled using sector duration 1.5 yr - which is the minimal sector duration atm?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, we are not analysing security here, but the cost overhead due to the new security params (ie, 12.8 x challenges).

But you are right that the cost difference between NI-PoRep and Interactive PoRep is maximum when sector duration is minimum.
I think it make sense con consider 3y as that is the most common duration we see in the network.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a reason why you think 3yr can be considered as the most common duration we see in the network?

Most SP set sector duration according to deal durations and not majority of the data clients are making 3yr deals (data from early Feb)
image

Copy link
Contributor

@irenegia irenegia Mar 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's on the x axis here?
are you saying majority of deals have 25000 epochs (~9 days)? @jennijuju
For the analysis we need to consider the avarage lifetime of the sector (ie, initial commitment + all the extensions).

FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
NI-PoRep has little downside with respect to the status quo: it removes PreCommit at the cost of augmenting C2 (ie SNARK generation) costs, which would result in a limited cost increase looking at onboarding costs as a whole). Indeed, NI-PoRep requires 12.8x more PoRep Challenges, which translates into an 12.8x SNARK proving overhead. We analyzed how this SNARK computation overhead affects overall costs. The conclusion is that considering PC1+PC2+C1+C2 and storage costs (i.e. not considering maintenance costs), a NI-PoRep sector with 128 bits of security is 5% more expensive than an Interactive PoRep sector when sector duration is 3y. See full analysis [here](../resources/fip-xxx-niporep/NIPoRep_CostAnalysis.pdf).

The new onboarding method `ProveCommitSectorsNI` is restricted to CC-sectors. We decided for this design for the following reasons:
1. We believe that the main users of NI-PoRep will be SaaS Providers, which will use NI-PoRep for CC sectors anyway. In the SaaS scenario, the flow where the SP ships the data over to an SaaS Provider and then gets back the sealed data (replica) seems more complex (and therefore more expensive in practice) than the flow where the SaaS Provider distributes CC sectors and then the SP can snap the data later. This should be especially true after [FIP0082](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0082.md) (SuperSnap) will be deployed in the network.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
1. We believe that the main users of NI-PoRep will be SaaS Providers, which will use NI-PoRep for CC sectors anyway. In the SaaS scenario, the flow where the SP ships the data over to an SaaS Provider and then gets back the sealed data (replica) seems more complex (and therefore more expensive in practice) than the flow where the SaaS Provider distributes CC sectors and then the SP can snap the data later. This should be especially true after [FIP0082](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0082.md) (SuperSnap) will be deployed in the network.
1. We believe that the main users of NI-PoRep will be SaaS Providers, which will use NI-PoRep for CC sectors anyway. In the SaaS scenario, the flow where (1) the SP ships the data over to an SaaS Provider and then gets back the sealed data (replica) seems more complex (and therefore more expensive in practice) than the flow where the SaaS Provider distributes CC sectors and then the SP can snap the data later; (2) Data client can technically transfer the data directly to SaaS, however, requiring clients needs to find both trustworthy SaaS and SP adds on undesired overhead. This should be especially true after [FIP0082](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0082.md) (SuperSnap) will be deployed in the network.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

an add-on based on my understanding - feel free to accept / modify / or reject

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is "data client" a common term?
I don't want to lead to confusion using it

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think so?

FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
FIPS/fip-xxxx-ni-porep.md Outdated Show resolved Hide resolved
@filecoin-project filecoin-project deleted a comment from jennijuju Mar 6, 2024
@jennijuju
Copy link
Member

Thank you @irenegia for addressing the feedbacks! This LGTM as the first Draft! Please update the fip doc with the fip number and add it to the readme before merging!

@irenegia
Copy link
Contributor

Thank you @irenegia for addressing the feedbacks! This LGTM as the first Draft! Please update the fip doc with the fip number and add it to the readme before merging!

done!

@anorth anorth mentioned this pull request Mar 11, 2024

NI-PoRep will implement the batch balancer currently in place. See [FIP-0013](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0013.md) for details.


Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a new parameter value for this will need to be calculated, otherwise aggregated NI-PoRep proofs could end up too much more expensive than non-aggregated ones. Measuring this can be a TODO before last call.

@anorth
Copy link
Member

anorth commented Mar 12, 2024

Please add to README in this PR before merging.

@irenegia
Copy link
Contributor

Please add to README in this PR before merging.

done

@jennijuju jennijuju merged commit e719f22 into filecoin-project:master Mar 16, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: ☑️ Done (Archive)
Development

Successfully merging this pull request may close these issues.

8 participants