Community Review of IPFS Force Allocator

[ipfsforcezuofu]

Allocator Application filecoin-project/notary-governance#1034

We sincerely apply for 10P to address the unfulfilled DC demand from the first round of assignment and to meet the upcoming demand from clients. Thank you.

[filecoin-watchdog]

Allocator Application
Compliance Report

All of the SPs have a retrieval rate below 75%.
5 out of 8 SPs have a retrieval rate of 0%.

Example 1:
The allocator in its applications caused a massive amount of confusion, both with its mistakes and inaccuracy in enforcing client information.
The client requested DC from the allocator at the beginning of August. After a series of incomprehensible agreements regarding the final amount of DC that would be granted to the client, the thread was moved to a separate issue on github (issue 8) due to technical problems.
After creating a new issue, the DC requested was given differently (even assuming that the client deducted the DC already granted, it remains unclear why in issue 8 he requested 2 PiB, while in issue 2 it was 5 PiB, of which 1 PiB was granted and used). At the same time, in issue 2, the client explicitly requested an allocation of another 4 PiB (and not 2).

Expanding on the above:

• The client requested 5 PiB in issue 2, of which - according to the allocator's application - 0.3 PiB was calculated as the first tranche,
• The allocator asked for KYC and suddenly decided that since it was a new client, he would grant a maximum of 1 PiB. based on which he calculated a new tranche schedule (60 TiB, according to the allocator's application)
  Instead of sticking to the established schedule, he mistakenly granted 1 PiB as the first tranche.
• Having realized his mistake, he decided to do compliance checks according to the originally established schedule. Could the allocator justify what would change this while DC had already been granted? If it was a mistake, the allocator should have contacted the governance team to withdraw the granted DC.

Apart from that, all SPs of this client have a retrieval rate of <5%.

Example 2
The allocator did better with the next client, but the client claims to be using 2 SPs, while the allocator application claims to require 5 replicas and SPs each. Has the allocator tried to clarify this with the client? The GitHub issue does not provide an answer to this question.

Side note: Why has the client asked for 3PiB, while they claim one copy will be 250TiB and there will be 4 replicas? It gives 1 PiB; what is the extra 2 PiB needed for?

[ipfsforcezuofu]

Question:

All of the SPs have a retrieval rate below 75%.
5 out of 8 SPs have a retrieval rate of 0%.

Answer:

Based on the confirmations from both clients in the following screenshots, most of their SPs are using DDO to place orders. As I understand, Spark will support retrievability for DDO by the end of October.

Expaination for Example 1:

I apologize for the confusion described in Example 1 due to my unfamiliarity with the FIL+ process and tools. I'm pleased to hear the positive feedback on Example 2, indicating improvement with the second client. To address the issues mentioned in Example 1, I will reiterate the entire distribution process step by step to show the background of my operation.

1. When I received the 5PiB request from the client, I quickly calculated the tranche and informed them. Upon realizing it was inappropriate to assign such a large DC amount to a new client, I adjusted it from 5PiB to 1PiB in the allocator repository.
2. When I logged into the DC distribution tool, I saw the request was still at 5PiB. I initially thought the first step should be to adjust it to 1PiB. However, I realized my mistake after confirming with the Ledger and seeing the result of the first tranche. I immediately contacted the client online and offline to ensure compliance. I was unaware that I could approach the government team to revoke the grant. Nevertheless, I monitored the situation and frequently communicated with the client to ensure compliance. I’m pleased that the outcome was positive; the report stated, "Data replication looks healthy. No CID sharing has been observed." The low retrieval rate was expected due to DDO ordering.
3. The client requested the remaining 4PiB after using 1PiB. Considering I couldn't allocate the entire DC of my allocator to a single client, I agreed to an additional 2PiB. After two unsuccessful refill attempts, I thought a new issue could be able to fix the issue for the second round of grants. To service the client promptly, I asked them to open a new issue without consulting the government team.
4. While attempting to refill the second tranche in the new issue, the distribution tool displayed an error message about the unverified address. I contacted the support team and opened issue number 72. In the mean time, I found the client address used in the previous issue worked again. After aligning with the client to ensure an uninterrupted process, I reopened the previous issue and refilled the remaining tranche.

The entire process of servicing client one caused many confusion. However, I'm pleased that I maintained compliance throughout. The key lesson I learned is to reach out to the government team or technical support for any uncertainties in the future. The outcome with the second client serves as a good example.

Question:

Example 2:
The allocator did better with the next client, but the client claims to be using 2 SPs, while the allocator application claims to require 5 replicas and SPs each. Has the allocator tried to clarify this with the client? The GitHub issue does not provide an answer to this question.

Answer:

To meet my application requirement of 5 replicas, the client provided 5 SPs, but only 2 were utilized. I didn't challenge the client, as I thought the requirement might be too high. I'm wondering if it can be adjusted.

Question:

Example 2:
Side note: Why has the client asked for 3PiB, while they claim one copy will be 250TiB and there will be 4 replicas? It gives 1 PiB; what is the extra 2 PiB needed for?

Answer:

I escalated your question to the client on GitHub, and here is their reply:

[galen-mcandrew]

Based on the investigation and details provided, we are seeing mixed compliance results. Some areas for this allocator to focus on should include:

• Verifying public client application details in GitHub, and providing diligence evidence for future bookkeeping investigations
• SP lists before and after allocations, with explanations from clients when there are changes made
• Investigating overall dataset size, replicas, and total DataCap requested calculations to avoid unnecessary sector padding
• Encouraging/requiring clients to work with SPs that are showing strong compliance and retrieval rates, rather than waiting for future DDO updates. For example, we are seeing increasing numbers of SPs that are consistently reporting over 75% retrieval rates

Given this review, we are requesting that the allocator verify that they will uphold all aspects & requirements of their initial application, with some extra focus on these areas. If so, we will request an additional 5PiB of DataCap from RKH, to allow this allocator to show increased diligence and alignment.

[ipfsforcezuofu]

@galen-mcandrew
Thank you for your comments. We will take the lessons learned from previous practices to improve our compliance performance, particularly in the areas highlighted in this report.

[Kevin-FF-USA]

DataCap was refilled.
https://datacapstats.io/notaries/f03019856