FIP-0083: add piece information to claim event #955
Conversation
rvagg
requested review from
momack2,
arajasek,
jennijuju,
kaitlin-beegle,
anorth,
raulk,
jsoares and
TippyFlitsUK
as code owners
|
Discussion of the motivation etc moved to #754 (comment) |
|
The motivation and the direction SGTM (responded in the thread linked by anorth above) - however, i am not convinced yet that one will be able to identify an unique entry by just provider ID, client ID, piece size and piece CID across verifreg and miner events. Even if someone get SectorNumber from Claim, it is still not an unique identifier (even tho the chances are low, but its possible one store the same piece is the same sector). What imho would be more efficient, could be having allocation ID in the sector events, with that toolings can quickly identify and update the verifreg table. |
|
Synced with Alex and he reminded me we have aligned to do a followup FIP in general (which rod told me too), so should we close this PR then? |
|
will close, but reluctantly, I think we're going to be bothered by the lack of piece (and probably sector, which is still missing in this PR) after nv22 just for basic data onboarding metrics |
Summary
Add piece information to the new builtin actor events (FIP-0083)
"claim"Background
Original discussion thread that lead toward minimalism in these events can be found here: #754 (comment)
Further specific discussion in this thread: #754 (comment) - although note the ambiguity in that discussion about including pieces ("if we are including pieces CIDs in the events, which sounds like we are - then I agree!"). Piece information is currently only in
"sector-activated"and"sector-updated".Reasoning for change
A primary impetus (as can be seen in discussion OP) for this first batch of builtin actor events was to provide visibility into information which is either no longer on chain, or otherwise difficult to to ascertain in the wake of a direct data onboarding workflow (FIP-0076).
We are now in the position of attempting to help a particular group of stakeholders with continuity post nv22: those interested in analytics and other kinds of Filecoin observability, who could previously rely on singular paths to specific observability needs, but post nv22 (post FIP-0076), now have a difficult time getting insight specifically to data onboarding concerns.
Consider the following analytics style questions:
These questions are all answerable, but they will require looking at, and keeping an ongoing record of, both state and actor events (either directly via API or in message receipts).
"sector-activated"and"sector-updated"currently includes piece information because it's not stored on chain, the only other place it appears is in the original messages from an SP; although there's some ambiguity between message and successful activation so these events are likely the only place this information is available for onboarding that doesn't touch the market actor.However, discerning which pieces are claiming DataCap is more difficult. The
"claim"event tells you the ClaimID (formerly AllocationID of the same), the client address and provider address. But it doesn't tell you which piece, or what size. You need to follow a"claim"event with an inspection of state for that ClaimID to find this information. (Figuring out the actual lifetime of the piece in the particular sector is another matter which also requires paying attention to events, adding new complications, but this is why additional information about the claim, such as the term, is less useful for the event).State inspection needs to be done soon after the event is emitted, because the claim doesn't necessarily persist in state since they can be optionally cleaned once the claim is no longer valid (the timeframe for clearing is detached beyond the claim or sector's active lifetime and is optional if someone wants to pay gas to clear it).
This all means that an observer that's primarily interested in questions of data onboarding over time is required to collect events and either: (1) capture claim->piece information at the time of
"claim"events from current state or (2) enough historical state to be able to query it at the particular tipset (or soon after) any"claim"event they are considering.Those performing broad analytics questions, like Starboard / Lily, are likely going to be capturing enough information to be (potentially) performing this complicated dance successfully. Smaller entities that have focused questions regarding data onboarding are going to have more difficulty, either in process or in storage needs.
Therefore I suggest that adding piece information to
"claim"("allocation"could be included here for symmetry but its utility isn't as strong, and we're already newly forcing gas onto the client here so maybe we don't need to increase it) will make this problem facing us with nv22 a little bit more straightforward. It's already going to be quite painful and error prone, but I believe this disentangles it enough that answering focused data-onboarding questions should be straightforward without having to round-trip to state as it exists at the time of the events. This will incur additional gas cost, although it's only two additional encoded log entries per event and FIP-0083 already demonstrated that the overall gas impact of events is very small.