| fip | title | author | discussions-to | status | type | category | created |
|---|---|---|---|---|---|---|---|
0083 |
Add built-in actor events |
Aarsh Shah (@aarshkshah1992) |
Final |
Technical (Core) |
Core |
2023-11-27 |
This FIP defines and proposes the implementation of a meaningful subset of fire-and-forget externally observable events that will be emitted by the built-in Verified Registry, Storage Market and Storage Miner Actors. These enable external agents (henceforth referred to as "clients") to observe and track a specific subset of on-chain datacap allocation/claims, deal lifecycle and sector lifecycle state transitions.
While this FIP explicitly delineates the implementation of events tied to specific transitions, it does not encompass the full spectrum of potential built-in actor events. The introduction and integration of additional built-in actor events will be addressed in subsequent FIPs.
FIP 0049 introduced and implemented the ability for actors to emit fire-and-forget externally observable events during execution.
This FIP aims to be the first of many for implementing events in the built-in actors. This specific FIP enumerates, defines and proposes the implementation of a subset of events for datacap allocation/claims, deal lifecycle and sector lifecycle transitions in the Verified Registry, Storage Market and Storage Miner Actors.
The events introduced in this FIP will enable clients such as network monitoring tools, block explorers, SP <> client deal brokering systems, browser clients, filecoin blockchain oracles etc. to rely on these events as a source of truth for sourcing information about the specific subset of on-chain activity and transitions captured by these events.
Events are emitted by the Verified Registry Actor for each verifier balance update, each allocation created, expired or claimed and for each claim that is updated or removed.
Events are emitted by the Market Actor for each deal that is published, activated, terminated or completed successfully.
Events are emitted by the Miner Actor for each sector that is precommitted, activated (provecommited), updated (SnapDeals) or terminated by an SP.
While FEVM smart contracts can already emit events, built-in actors are still eventless. This FIP is the first step towards fixing the external observability of built-in actors.
Filecoin network observability tools, block explorers, SP monitoring dashboards, deal brokering software, etc. need to rely on complex low-level techniques such as state tree diffs to source information about on-chain activity and state transitions. Emitting the select subset of built-in actor events defined in this FIP will make it easy for these clients to source the information they need by simply subscribing to the events and querying chain state.
Please see FIP-0049 for a detailed explanation of the FVM event schema and the possible values each field in the event schema can take. The interpretation of the flags field in each event entry is as follows:
- {"flags": "0x03"} ⇒ Index both key and value of the event entry
- {"flags": "0x02"} ⇒ Index only the value of the event entry
- {"flags": "0x01"} ⇒ Index only the key of the event entry
All values in the event payloads defined below are encoded with CBOR and so the codec field for each event entry will be set to 0x51. The codec field is left empty in the payloads defined below only for the sake of brevity.
The FVM currently only supports event values encoded with the raw encoding. This FIP adds
support for CBOR encoding (0x51), and built-in actor event values will be encoded with CBOR.
Note that the "bigint" CBOR encoding format used below for token amount fields is the same as is
used for encoding bigints on the Filecoin chain: a byte array representing a big-endian unsigned
integer, compatible with the Golang big.Int byte representation, with a 0x00 (positive) or
0x01 (negative) prefix; with a zero-length array representing a value of 0.
This event is emitted when the balance of a verifier is updated in the Verified Registry Actor.
The event payload is defined as:
| flags | key | value |
|---|---|---|
| Index Key + Value | "$type" | "verifier-balance" (string) |
| Index Key + Value | "verifier" | <VERIFIER_ACTOR_ID> (int) |
| Index Key + Value | "client" | <DATACAP_HOLDER_ACTOR_ID> (optional int) (only present when datacap allocated to a client) |
| Index Key | "balance" | <VERIFIER_DATACAP_BALANCE> (bigint) |
This event is emitted when a verified client allocates datacap to a specific data piece and storage provider.
The event payload is defined as:
| flags | key | value |
|---|---|---|
| Index Key + Value | "$type" | "allocation" (string) |
| Index Key + Value | "id" | <ALLOCATION_ID> (int) |
| Index Key + Value | "client" | <CLIENT_ACTOR_ID> (int) |
| Index Key + Value | "provider" | <SP_ACTOR_ID> (int) |
| Index Key + Value | "piece-cid" | <PIECE_CID> (cid) |
| Index Key | "piece-size" | <PIECE_SIZE> (int) |
| Index Key | "term-min" | <TERM_MINIMUM> (int) |
| Index Key | "term-max" | <TERM_MAXIMUM> (int) |
| Index Key | "expiration" | <EXPIRATION_EPOCH> (int) |
This event is emitted when a datacap allocation that is past its expiration epoch is removed.
The event payload is defined as:
| flags | key | value |
|---|---|---|
| Index Key + Value | "$type" | "allocation-removed" (string) |
| Index Key + Value | "id" | <ALLOCATION_ID> (int) |
| Index Key + Value | "client" | <CLIENT_ACTOR_ID> (int) |
| Index Key + Value | "provider" | <SP_ACTOR_ID> (int) |
| Index Key + Value | "piece-cid" | <PIECE_CID> (cid) |
| Index Key | "piece-size" | <PIECE_SIZE> (int) |
| Index Key | "term-min" | <TERM_MINIMUM> (int) |
| Index Key | "term-max" | <TERM_MAXIMUM> (int) |
| Index Key | "expiration" | <EXPIRATION_EPOCH> (int) |
This event is emitted when a client allocation is claimed by a storage provider after the corresponding verified data is provably committed to the chain.
The event payload is defined as:
| flags | key | value |
|---|---|---|
| Index Key + Value | "$type" | "claim" (string) |
| Index Key + Value | "id" | <CLAIM_ID> (int) |
| Index Key + Value | "client" | <CLIENT_ACTOR_ID> (int) |
| Index Key + Value | "provider" | <SP_ACTOR_ID> (int) |
| Index Key + Value | "piece-cid" | <PIECE_CID> (cid) |
| Index Key | "piece-size" | <PIECE_SIZE> (int) |
| Index Key | "term-min" | <TERM_MINIMUM> (int) |
| Index Key | "term-max" | <TERM_MAXIMUM> (int) |
| Index Key | "term-start" | <TERM_START> (int) |
| Index Key + Value | "sector" | <SECTOR_ID> (int) |
This event is emitted when the term of an existing allocation is extended.
The event payload is defined as:
| flags | key | value |
|---|---|---|
| Index Key + Value | "$type" | "claim-updated" (string) |
| Index Key + Value | "id" | <CLAIM_ID> (int) |
| Index Key + Value | "client" | <CLIENT_ACTOR_ID> (int) |
| Index Key + Value | "provider" | <SP_ACTOR_ID> (int) |
| Index Key + Value | "piece-cid" | <PIECE_CID> (cid) |
| Index Key | "piece-size" | <PIECE_SIZE> (int) |
| Index Key | "term-min" | <TERM_MINIMUM> (int) |
| Index Key | "term-max" | <TERM_MAXIMUM> (int) |
| Index Key | "term-start" | <TERM_START> (int) |
| Index Key + Value | "sector" | <SECTOR_ID> (int) |
This event is emitted when an expired claim is removed by the Verified Registry Actor.
The event payload is defined as:
| flags | key | value |
|---|---|---|
| Index Key + Value | "$type" | "claim-removed" (string) |
| Index Key + Value | "id" | <CLAIM_ID> (int) |
| Index Key + Value | "client" | <CLIENT_ACTOR_ID> (int) |
| Index Key + Value | "provider" | <SP_ACTOR_ID> (int) |
| Index Key + Value | "piece-cid" | <PIECE_CID> (cid) |
| Index Key | "piece-size" | <PIECE_SIZE> (int) |
| Index Key | "term-min" | <TERM_MINIMUM> (int) |
| Index Key | "term-max" | <TERM_MAXIMUM> (int) |
| Index Key | "term-start" | <TERM_START> (int) |
| Index Key + Value | "sector" | <SECTOR_ID> (int) |
The Market Actor emits the following deal lifecycle events:
This event is emitted for each new deal that is successfully published by a storage provider.
The event payload is defined as:
| flags | key | value |
|---|---|---|
| Index Key + Value | "$type" | "deal-published" (string) |
| Index Key + Value | "id" | <DEAL_ID> (int) |
| Index Key + Value | "client" | <STORAGE_CLIENT_ACTOR_ID> (int) |
| Index Key + Value | "provider" | <STORAGE_PROVIDER_ACTOR_ID> (int) |
The deal activation event is emitted for each deal that is successfully activated.
The event payload is defined as:
| flags | key | value |
|---|---|---|
| Index Key + Value | "$type" | "deal-activated" (string) |
| Index Key + Value | "id" | <DEAL_ID> (int) |
| Index Key + Value | "client" | <STORAGE_CLIENT_ACTOR_ID> (int) |
| Index Key + Value | "provider" | <STORAGE_PROVIDER_ACTOR_ID> (int) |
FIP-0074 ensures that terminated deals are processed immediately in the OnMinerSectorsTerminate method
rather than being submitted for deferred processing to the market actor cron job. That change will make
this event available to clients.
The deal termination event is emitted by the market actor cron job when it processes deals that were marked
as terminated by the OnMinerSectorsTerminate method.
The event payload is defined as:
| flags | key | value |
|---|---|---|
| Index Key + Value | "$type" | "deal-terminated" (string) |
| Index Key + Value | "id" | <DEAL_ID> (int) |
| Index Key + Value | "client" | <STORAGE_CLIENT_ACTOR_ID> (int) |
| Index Key + Value | "provider" | <STORAGE_PROVIDER_ACTOR_ID> (int) |
This event is emitted when a deal is marked as successfully complete by the Market Actor cron job. The cron job will deem a deal to be successfully completed if it is past it’s end epoch without being slashed.
This event is not usable as it is emitted from a cron job. However, we still emit the event here for completeness. FIP-0074 ensures that the processing of completed deals is done as part of a method called by the Storage Provider thus making this event available to clients and also to ensure that SPs pay the gas costs of processing deal completion and event emission. This applies to new deals made post landing FIP-0074. For deals made before FIP-0074 lands, this event is not usable.
The event payload is defined as:
| flags | key | value |
|---|---|---|
| Index Key + Value | "$type" | "deal-completed" (string) |
| Index Key + Value | "id" | <DEAL_ID> (int) |
| Index Key + Value | "client" | <STORAGE_CLIENT_ACTOR_ID> (int) |
| Index Key + Value | "provider" | <STORAGE_PROVIDER_ACTOR_ID> (int) |
The Miner Actor emits the following sector lifecycle events:
The sector pre-committed event is emitted for each new sector that is successfully pre-committed by an SP.
The event payload is defined as:
| flags | key | value |
|---|---|---|
| Index Key + Value | "$type" | "sector-precommitted" (string) |
| Index Key + Value | "sector" | <SECTOR_NUMBER> (int) |
This event is emitted for each pre-committed sector that is successfully activated by a storage provider. For now, sector activation corresponds 1:1 with prove-committing a sector but this can change in the future.
The event payload is defined as:
| flags | key | value |
|---|---|---|
| Index Key + Value | "$type" | "sector-activated" (string) |
| Index Key + Value | "sector" | <SECTOR_NUMBER> (int) |
| Index Key + Value | "unsealed-cid" | <SECTOR_COMMD> (nullable cid) (null means sector has no data) |
| Index Key + Value | "piece-cid" | <PIECE_CID> (cid) |
| Index Key | "piece-size" | <PIECE_SIZE> (int) |
piece-cidandpiece-sizeis included for each piece in the sector, i.e. they may appear zero or more times in the event payload but always as a pair.
This event is emitted for each CC sector that is updated to contained actual sealed data.
The event payload is defined as:
| flags | key | value |
|---|---|---|
| Index Key + Value | "$type" | "sector-updated" (string) |
| Index Key + Value | "sector" | <SECTOR_NUMBER> (int) |
| Index Key + Value | "unsealed-cid" | <SECTOR_COMMD> (nullable cid) (null means sector has no data) |
| Index Key + Value | "piece-cid" | <PIECE_CID> (cid) |
| Index Key | "piece-size" | <PIECE_SIZE> (int) |
piece-cidandpiece-sizeis included for each piece in the sector, i.e. they may appear zero or more times in the event payload but always as a pair.
The sector termination event is emitted for each sector that is marked as terminated by a storage provider.
The event payload is defined as:
| flags | key | value |
|---|---|---|
| Index Key + Value | "$type" | "sector-terminated" (string) |
| Index Key + Value | "sector" | <SECTOR_NUMBER> (int) |
The legacy (pre FIP-0076) sector activation and sector update methods and flows in the Miner Actor need access to the piece manifests for the sector data to build the sector activation and sector update event payloads described above. This information is currently persisted in the Market Actor.
Since these legacy flows already call the Market Actor BatchActivateDeals(method 6) method to activate the deals in the sectors,
the return type of this method has been changed to include the piece manifests for the sector data.
The updated BatchActivateDealsResult type is defined as:
type BatchActivateDealsResult {
/// Status of each sector grouping of deals.
activation_results: BatchReturn,
/// Activation information for the sector groups that were activated.
activations: Vec<SectorDealActivation>,
}
type SectorDealActivation {
/// Information about each deal activated.
activated: Vec<ActivatedDeal>,
/// Unsealed CID computed from the deals specified for the sector.
/// A None indicates no deals were specified, or the computation was not requested.
unsealed_cid: Option<Cid>
}
type ActivatedDeal {
client: ActorID,
allocation_id: AllocationID, // NO_ALLOCATION_ID for unverified deals.
data: Cid, // "piece-cid"
size: PaddedPieceSize // "piece-size"
}
This is not a problem for the sector activation and sector update methods and flows introduced in FIP-0076 as those methods already have access to the piece manifests for the sector data in their input params.
Richer and more detailed payloads for the events were initially considered. However, a large event payload increases the event emission gas costs are borne by users of the built-in actor methods. The payload of an event should, therefore, be as minimal as possible to reduce costs. However, the event payloads should still have enough information for clients consuming the events to determine if they are interested in reacting to the event (e.g. by querying chain state to obtain additional information about the transition).
The Verified Registry Event payloads have the allocation/claim IDs and client and provider Actor IDs. This enables users to filter the events by the client/provider whose claim/allocation they are interested in and then query the chain state with the corresponding allocation/claim ID to get more information.
Once a deal is published, clients have access to the dealId of the deal. All Market Actor events except
for the "deal published" event have the dealId in their payload, which clients can use to filter for events
they are interested in and then query the chain state for more information.
All Market Actor events have the client and provider Actor IDs in the event payload. This enables users
to filter these events by the specific deal making parties they are interested in and then query the chain
state with the corresponding dealId to get more information.
However, note that the dealId is not known to the storage client before the deal is actually published as it is generated by
the storage provider during the call to PublishStorageDeals.
Note that cron jobs do not return message receipts containing the emitted events back to the user.
Therefore, the Market Actor deal-terminated event will not be usable as it stands today as the deals
are terminated by the cron job and so the event is also emitted from the cron job.
However, once FIP-0074 is implemented,
this event will be emitted from the OnMinerSectorsTerminate method itself. This will make the event available to
clients and the gas cost for this event will be paid by the OnMinerSectorsTerminate method.
The above is also true for the deal-completed event. Deal completion is currently processed by a
cron job and so the event will not be usable by clients.
FIP-0074 will ensure that the processing of completed deals is done as part of a method called by the
Storage Provider thus making this event available to clients
All Miner Actor Events will already have the Miner Actor ID in the emitter field of the event.
So, we only need to include the sector number in the event payload for Miner Actor events to enable
clients to filter Miner Actor events by the Miner/Sector they are interested in.
An SP can activate a single sector by using the ProveCommitSector method. As of today, this method
schedules a cron job to actually activate the sector which means that the sector activation event will be
emitted by the cron job. This makes the event un-usable as events emitted by a cron job are not included
in the message receipt. However, we still emit the sector activation event in the Miner Actor for
completeness. FIP-0076 will ensure that sector activation is actually done as part of the
ProveCommitSector method thus making this event available to clients and also to ensure that SPs
actually pay the cost of sector activation and event emission.
There is an edge case we need to consider for the sector-terminated event in the Miner Actor.
Some sectors can also be marked as terminated by the Miner Actor cron job as the number of terminations
that can be processed immediately by the TerminateSectors method are rate-limited.
The events emitted for these terminations will not be available to the client because they are
emitted from a cron job.
Once FIP-0076 is implemented,
sectors can be activated with data that is not associated with a built-in market actor deal. This means that clients may
not be able to obtain additional information about this data by querying the Market Actor state.
Therefore, the sector activation events contains the unsealed_cid(COMMD) and the piece_cid and
piece_size for each piece in the sector as this information will be useful to aggregators, indexers etc.
This proposal does not remove or change any exported APIs, nor change any state schemas. It is backwards compatible for all callers. Since it requires a change to actor code, a network upgrade is required to deploy it.
The actor events introduced in this FIP are fire and forget. No event is persisted in the actor state and clients can choose not to consume any/all of the events.
Events emitted by built-in actors do form a part of the MessageReceipt chain data structure via
the existing events_root field on MessageReceipt. While this field is set to empty today for
built-in actor method calls, this field will contain a value once built-in actors start emitting events.
This FIP proposes adding CBOR encoding for event values. Clients that are interested in built-in actor events will need to upgrade to understand CBOR.
However, this FIP does not propose changing the existing event schema or changing any of the chain
data structures including the MessageReceipt structure. So, this change should be completely backward
compatible.
All pre-existing unit tests in the Verified Registry, Market and Miner Actors should expect and observe the emission of the events defined in this proposal wherever applicable.
Existing integration tests should also expect and observe the emission of events defined in this proposal wherever applicable.
This proposal does not introduce any reductions in security.
Even though the event payloads have been kept minimal to minimise the gas costs for emitting events, this proposal does increase the gas cost of methods in the Verified Registry, Miner and Market Actors that emit the events defined in this FIP.
However, the gas costs for emitting these events is a trivial fraction of the overall gas costs for the methods that emit these events.
For the ProveCommitSectors3 method, the cost of emitting the sector activation event for a single sector with one piece is
around ~28000 gas units which is ~0.05% of the total gas cost of the method. For the ProveReplicaUpdates3 method,
the cost of emitting the sector update event for a single sector with one piece is ~28000 gas units which is around 0.077%
of the gas cost of verifying a single replica update. For the Market Actor PublishStorageDeals method, the cost of emitting
the published event for a single deal is ~24000 gas units which is ~0.06% of the total gas cost of the method.
While this increased gas cost is borne by the users of the methods, the benefit of the emitted events is enjoyed by networking monitoring tools, network accounting tools, block explorer tools and other external agents that provide some value-added services to the Filecoin network. Improved implementation of these tools will indirectly benefit the network users.
This FIP will enable tools dedicated to network observability, monitoring, and accounting, as well as other external agents, to transition away from depending on highly-coupled, internal and low-level mechanisms such as state tree diffs/message-replays to source information about built-in actors activity and state transitions. They can instead use the events defined in this FIP as a source of truth to obtain the information they need (*for information captured by the specific subset of built-in actor events defined in this FIP).
Clients will still have to rely on pre-existing mechanisms for observability of transitions not captured by events defined in this FIP.
Validating nodes will have to store event data to make that data available to clients just as they store message receipts. This will increase the storage requirements for the validating nodes but event payloads have been design to be as minimal as possible to keep the storage requirement to a bare minimum.
Implementation of this proposal is in progress on the following feature branch: https://github.com/filecoin-project/builtin-actors/tree/integration/actor-events
Copyright and related rights waived via CC0.