Skip to content
This repository has been archived by the owner on Jul 18, 2024. It is now read-only.

[DataCap Application] PiKNiK - NEXRAD Slingshot Dataset 1/4 #432

Closed
jessie8o8 opened this issue Jun 24, 2022 · 48 comments
Closed

[DataCap Application] PiKNiK - NEXRAD Slingshot Dataset 1/4 #432

jessie8o8 opened this issue Jun 24, 2022 · 48 comments
Assignees

Comments

@jessie8o8
Copy link

jessie8o8 commented Jun 24, 2022

Large Dataset Notary Application

To apply for DataCap to onboard your dataset to Filecoin, please fill out the following.

Core Information

  • Organization Name: PiKNiK
  • Website / Social Media: piknik.com
  • Total amount of DataCap being requested (between 500 TiB and 5 PiB): 5 PiB
  • Weekly allocation of DataCap requested (usually between 1-100TiB): 100 TiB
  • On-chain address for first allocation: f13csteap4jcgl3a65qtsvmblzvelp4sgivddtpqa

Please respond to the questions below by replacing the text saying "Please answer here". Include as much detail as you can in your answer.

Project details

Share a brief history of your project and organization

PiKNiK is the main client for the USC Shoah Foundation and has experience in allocation deals to other SPs. We work to help others learn how to store valuable data by hosting an accelerator program called ESPA. 

This application is the first of the four as the total datacap being asked is of 20 PiBs, thus having to split the datacap application into 4 parts

What is the primary source of funding for this project?

PiKNiK

What other projects/ecosystem stakeholders is this project associated with?

Slingshot competition hosted by Protocol Labs

Use-case details

Describe the data being stored onto Filecoin

Next Generation Weather Radar is a network of 160 high-resolution S-band Doppler weather radars that detect precipitation and wind patterns and movement.

Where was the data in this dataset sourced from?

Google Cloud Public Datasets, the Registry of Open Data on AWS, and Azure Open Datasets

Can you share a sample of the data? A link to file would work.

https://registry.opendata.aws/noaa-nexrad/

Confirm that this is a public dataset that can be retrieved by anyone on the Network (i.e., no specific permissions or access rights are required to view the data).

Confirmed

What is the expected retrieval frequency for this data?

Slingshot retrieval requirements will be followed

For how long do you plan to keep this dataset stored on Filecoin?

For the duration of Slingshot

DataCap allocation plan

In which geographies (countries, regions) do you plan on making storage deals?

Europe, United States, Australia or any other SPs that are interested.

How will you be distributing your data to storage providers? Is there an offline data transfer process?

Offline

How do you plan on choosing the storage providers with whom you will be making deals? This should include a plan to ensure the data is retrievable in the future both by you and others.

PiKNIK plans to deal with select SPs that are reputable in the community and are reliable such as SPXs, SPs that have prominent presence in the community, SPs that PiKNiK has successfully worked with before or using sites that rank reliability of SPs.

How will you be distributing deals across storage providers?

Hosting files online where storage providers can download them.

Do you have the resources/funding to start making deals as soon as you receive DataCap? What support from the community would help you onboard onto Filecoin?

Yes. Additional support would come from Slingshot support channels in Filecoin Slack if needed.
@large-datacap-requests
Copy link

Thanks for your request!
Everything looks good. 👌

A Governance Team member will review the information provided and contact you back pretty soon.

@jessie8o8
Copy link
Author

The rest of the data cap applications for this dataset:
[DataCap Application] - PiKNiK - NEXRAD Slingshot Dataset 2/4 #433
[DataCap Application] - PiKNiK - NEXRAD Slingshot Dataset 3/4 #434
[DataCap Application] - PiKNiK - NEXRAD Slingshot Dataset 4/4 #435

@dkkapur dkkapur assigned dkkapur and unassigned galen-mcandrew Jul 6, 2022
@dkkapur
Copy link
Collaborator

dkkapur commented Jul 6, 2022

Datacap Request Trigger

Total DataCap requested

5PiB

Expected weekly DataCap usage rate

400TiB

Client address

f13csteap4jcgl3a65qtsvmblzvelp4sgivddtpqa

@large-datacap-requests
Copy link

large-datacap-requests bot commented Aug 4, 2022

DataCap Allocation requested

Multisig Notary address

f02049625

Client address

f13csteap4jcgl3a65qtsvmblzvelp4sgivddtpqa

DataCap allocation requested

50TiB

@dkkapur
Copy link
Collaborator

dkkapur commented Aug 4, 2022

@jessie8o8 we figured out the bug wrt the trigger not working above. Origin Storage team in Slack flagged that this may be because at the top, the amounts requested are in units "PiBs" and "TiBs" rather than "PiB" and "TiB".

However - Slingshot v2.8 ended. Are you still planning on storing this dataset outside the scope of Slingshot?

I'm going to close out the associated issues in the meantime, and let's align on your next steps here and discuss the right course of action accordingly. Thank you.

@jessie8o8
Copy link
Author

Hi @dkkapur. Thanks for figuring out the bug. Noted for next time!

We still completely intend on storing the NEXRAD data under the slingshot vision as it is still useful data. We understand that the reward period has passed and will still commit to storing without the reward.

@dkkapur
Copy link
Collaborator

dkkapur commented Aug 4, 2022

@jessie8o8 ACK - can I request that we close out this application in that case and you open up another one that explicitly calls out your desire to store this as an open dataset regardless of Slingshot?

@xinaxu
Copy link
Contributor

xinaxu commented Jul 6, 2023

checker:manualTrigger

@filplus-checker-app
Copy link

DataCap and CID Checker Report Summary1

Retrieval Statistics

  • Overall Graphsync retrieval success rate: 14.01%
  • Overall HTTP retrieval success rate: 0.00%
  • Overall Bitswap retrieval success rate: 0.00%

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 63.69% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients2

⚠️ CID sharing has been observed. (Top 3)

Full report

Click here to view the CID Checker report.
Click here to view the Retrieval report.

Footnotes

  1. To manually trigger this report, add a comment with text checker:manualTrigger

  2. To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Copy link
Contributor

xinaxu commented Jul 6, 2023

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacedmzxgd752nbhhdnxosf4ypdyiayxzfjsqvmvxsfr2medcgvudfoa

Address

f13csteap4jcgl3a65qtsvmblzvelp4sgivddtpqa

Datacap Allocated

800.00TiB

Signer Address

f1k3ysofkrrmqcot6fkx4wnezpczlltpirmrpsgui

Id

f0cbccdc-93b4-4838-a4e1-980737ede88d

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedmzxgd752nbhhdnxosf4ypdyiayxzfjsqvmvxsfr2medcgvudfoa

Copy link

s0nik42 commented Jul 11, 2023

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceb7y5vh7two5qtbe36vm52zzpjwe3ggckna74sd75iofminay6s7u

Address

f13csteap4jcgl3a65qtsvmblzvelp4sgivddtpqa

Datacap Allocated

800.00TiB

Signer Address

f1wxhnytjmklj2czezaqcfl7eb4nkgmaxysnegwii

Id

f0cbccdc-93b4-4838-a4e1-980737ede88d

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceb7y5vh7two5qtbe36vm52zzpjwe3ggckna74sd75iofminay6s7u

@github-actions
Copy link

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

@github-actions github-actions bot added the Stale label Jul 22, 2023
@jamerduhgamer
Copy link

Datacap is currently being sealed.

@github-actions github-actions bot removed the Stale label Jul 23, 2023
@github-actions
Copy link

github-actions bot commented Aug 3, 2023

This application has not seen any responses in the last 10 days. This issue will be marked with Stale label and will be closed in 4 days. Comment if you want to keep this application open.

@github-actions github-actions bot added the Stale label Aug 3, 2023
@github-actions
Copy link

github-actions bot commented Aug 8, 2023

This application has not seen any responses in the last 14 days, so for now it is being closed. Please feel free to contact the Fil+ Gov team to re-open the application if it is still being processed. Thank you!

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Aug 8, 2023
@data-programs data-programs added the kyc verified User has passed KYC check label Aug 8, 2023
@data-programs
Copy link
Collaborator

KYC

This user’s identity has been verified through filplus.storage

@Sunnyiscoming
Copy link
Collaborator

Hello, @jessie8o8 per the filecoin-project/notary-governance#922 for Open, Public Dataset applicants, please complete the following Fil+ registration form to identify yourself as the applicant and also please add the contact information of the SP entities you are working with to store copies of the data.

This information will be reviewed by Fil+ Governance team to confirm validity and then the application will be allowed to move forward for additional notary review.

@jessie8o8
Copy link
Author

Hi @Sunnyiscoming, the form is complete and submitted.

Client f01870062 does not follow the datacap usage rules. More info here.
This application has been failing the requirements for 7 days.
Please take appropiate action to fix the following DataCap usage problems.

Criteria Treshold Reason
Shared data percent < 20% 20.95% of the clients data is shared with other clients. This should be less than 20%

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests