FIP-0071: Deterministic State Access (IPLD Reachability) #763

Stebalien · 2023-07-28T04:15:45Z

This FIP brings us a step closer to user-defined WebAssembly actors by introducing deterministic rules defining what "state" (IPLD blocks) an actor is and is not allowed to read. Specifically, this FIP specifies that an actor may only read state "reachable" from:

The actor's state-tree.
Parameters passed into the actor from other actors.
Blocks returned to the actor from other actors.

Where "reachable" means that the state can be "reached" by traversing IPLD links (CIDs) from the "roots" listed above.

Discussions: #764
WIP Implementation: filecoin-project/ref-fvm#1824

ZenGround0 · 2023-07-28T19:02:20Z

FIPS/fip-XXXX.md

+4. If the major type is 0 (integer), 1 (negative integer), or 7 (special), we continue.
+5. If the major type is 2 (byte string) or 3 (string), we seek forward "immediate value" bytes and continue. We do not validate the string's encoding.
+6. If the major type is 4 (array) we add the immediate value to the expected number of fields, and continue.
+7. If the major type is 5 (map) we add two times the immediate value to the expected number of fields, and continue. We do not or restrict the allowed key/value types, nor do we validate the order of keys, nor do we check for duplicates.


IIRC IPLD CBOR has a hard constraint that all map keys are strings. Is the idea that any block already linked will be valid IPLD CBOR so there's no special care needed to ensure that?

IIRC IPLD CBOR has a hard constraint that all map keys are strings. Is the idea that any block already linked will be valid IPLD CBOR so there's no special care needed to ensure that?

No, the idea is that the FVM doesn't really care one way or the other. Applications may care, but then its their job to check. The FVM checks the property it cares about.

(small rant)

I'd like to eventually move towards a world were we split the IPLD model into:

A "block" model. I.e., blocks, CIDs, and blocks that link to other blocks. This is the level of bitswap, blockstores, garbage collection, etc.

A "path" model that simply defines how to path through blocks. You need this for basic pathing, filesystems, etc.

A "data" model. This is the part that starts caring about map key types, ordering of map keys, etc. You need this if you want to write general-purpose algorithms over abstract data (e.g., graphsync).

One of the biggest challenges in IPLD has been agreeing on how formats map to the data model, especially as not all formats will nicely "fit" into the data model. My goal here is to say that fitting into the data model simply isn't necessary.

Of course, maybe we should use a codec that's not "dag cbor", but I don't think that's really necessary.

I'll add some of this to the design tradeoffs.

ZenGround0

Could you explain a bit more of the details and motivation behind the handle concept? What does this level of indirection help us with as opposed to just using cids directly as handles and merging block_create with block_link and block_open with block_read?

Are handles uniquely determined by cids or the same cid could have multiple handles within the same actor? Are handles for the same cid different in different actors? Do handles contain information about which actor's reachable set they originate from? What do handles look like in memory/serializaed as method parameters in send:send?

ZenGround0 · 2023-07-28T19:44:44Z

FIPS/fip-XXXX.md

+1. Take advantage of our existing Wasm gas accounting rather than manually charging for gas based on the CBOR's structure.
+2. Ensure that all implementations used the exact same wasm parsing and validation logic.
+
+However, this would have increased implementation complexity and introduced additional runtime costs (switching in and out of Wasm isn't free).


It would be very nice to use the same gas model to account for this. Do you have a quantitative estimate of how much worse this makes performance? For block_create it seems like we'd need no context switching since we already have the data for processing in WASM memory. Extra context switching shows up during block_open because only the handle need be sent back to WASM from syscall? So we'd pay for two writes to wasm memory once in block_open once in block_read?

Unfortunately, we do need to copy and jump every time because we can't trust the user's wasm instance. Basically:

Start a new wasm instance for processing the block. We could try to reuse instances here, but that may lead to "weird things". E.g., different costs depending on whether or not memory is already pre-allocated.

Jump into the wasm instance to allocate some memory for the block.

Jump back into the system to copy the block into the wasm instance's memory.

Jump back into the wasm instance to parse the block.

Finally, return to the system.

And then parse the CIDs emitted by the wasm instance.

But the main part really is complexity:

Doing this in wasm saves us from having to define 2 gas charges.

Doing this natively saves us from having to manage an additional wasm module, spin up an instance, define the syscalls for crossing back and forth, etc.

Basically, the current approach is both simpler and faster.

Stebalien · 2023-07-28T20:17:22Z

Could you explain a bit more of the details and motivation behind the handle concept? What does this level of indirection help us with as opposed to just using cids directly as handles and merging block_create with block_link and block_open with block_read?

I've added a link to https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0030.md#ipld-memory-model to the background section. Those syscalls aren't being introduced in this FIP (although I'm happy to discuss them here).

Basically, it comes down to the fact that:

We might want to "create" blocks without hashing them (hence returning a handle from create instead of returning a CID). This lets us send parameters/return values as IPLD blocks without hashing.
We won't know the size of an IPLD block when we first try to open it so we don't have a great way to pre-allocate memory for it. The current open/read API mimics the unix file API.

anorth

I'll be happy to give this an editorial 👍 when the abstract is filled in.

I note that test cases will also need to be specified before it can move to last call.

Stebalien · 2023-08-02T01:51:18Z

Ok, this is now ABG&T (all but gas and tests).

FIPS/fip-XXXX.md

Co-authored-by: Alex North <[email protected]>

fridrik01

oh had some comments pending for 2 weeks without noticing, sharing now

fridrik01 · 2023-07-31T10:08:04Z

FIPS/fip-XXXX.md

+1. `ipld::block_create(codec, data) -> handle` registers a new IPLD block with the FVM, returning a block "handle" (similar to a file descriptor).
+2. `ipld::block_open(cid) -> handle` "opens" the IPLD block referenced by the passed CID, returning a handle to the block.
+3. `ipld::block_read(handle, offset, length) -> data` reads data from an open block into the actor's memory.
+4. `ipld::block_link(handle, multihash_code, multihash_length)` creates a CID for the block referenced by the given block "handle", using the specified multihash code and length (currently limited to blake2b-256).


I think those signatures are missing some params, for example shouldn't block_read be:

ipld::block_read(handle, offset, obuf, max_length) reads block identified from handle into obuff which is in actors' memory

Technically, but I'm trying to convey the concept here more than the literal API (that's specified in the other FIP linked at the top of the spec). I'll document that.

fridrik01 · 2023-07-31T10:15:20Z

FIPS/fip-XXXX.md

+    1. If the CID uses the identity hash function (inlines a block), this function will not return the CID directly but will instead recursively parse the inlined block.
+    2. Otherwise, this function will emit the CID.
+2. Any "ignored CIDs" will be skiped and ignroed.
+3. Any other CIDs will cause this function to signal an error.


Did you consider ignoring them?

Good point. I've added a section to the discussion.

https://github.com/filecoin-project/FIPs/pull/763/files#diff-2554e48e24a96725aee3a2de915ff76596325561147c0dc73ac16225e5161c5eR245-R247

FIPS/fip-XXXX.md

Co-authored-by: Friðrik Ásmundsson <[email protected]>

kaitlin-beegle · 2023-08-16T19:48:23Z

FIPS/fip-XXXX.md

+## Security Considerations
+<!--All FIPs must contain a section that discusses the security implications/considerations relevant to the proposed change. Include information that might be important for security discussions, surfaces risks and can be used throughout the life cycle of the proposal. E.g. include security-relevant design decisions, concerns, important discussions, implementation-specific guidance and pitfalls, an outline of threats and risks and how they are being addressed. FIP submissions missing the "Security Considerations" section will be rejected. A FIP cannot proceed to status "Final" without a Security Considerations discussion deemed sufficient by the reviewers.-->
+
+This FIP primarily aims to increase the security of the FVM and guard against malicious and/or buggy wasm actors. However, as with any code:


nit, but I think the convention is capital "W". So, Wasm.

kaitlin-beegle

Approved editorial review.

Also, great work! This is a really well written FIP. No real editorial changes, and very simply stated.

jennijuju · 2023-08-22T04:30:13Z

FIPS/fip-XXXX.md

+
+We will call these "allowed CIDs".
+
+Additionally, FVM block link analysis will ignore CIDs of sealed and unsealed commitments, as long as the multihash _digest_ is less than 64 bytes:


FVM block link analysis will ignore CIDs of sealed and unsealed commitments

is the reason for specify these two cuz sealed and unsealed commitments are the only two fields in state that follows the same codecs ?

It's because:

We store such CIDs in the state already.

Sector state isn't "reachable" from actor code regardless.

Basically, it lets actors refer to sectors and pieces directly (by CID) without messing with the FVM complaining about such CIDs not being reachable. I'll update the text.

jennijuju · 2023-08-22T04:31:40Z

FIPS/fip-XXXX.md

@@ -0,0 +1,313 @@
+---
+fip: "<to be assigned>" <!--keep the qoutes around the fip number, i.e: `fip: "0001"`-->


jennijuju · 2023-08-22T04:33:17Z

(havent fully reviewed yet - tho @Stebalien @fridrik01 I have assigned FIP-0071 to this FIP and feel free to merge with updated FIP number given you have gotten 2 editorial ✅ already, further updates can be made in different PRs

Stebalien · 2023-08-22T05:23:30Z

Review addressed and FIP# assigned.

jennijuju · 2023-08-22T05:26:06Z

Let’s also add it to the readme table - please merge once that’s in

Deterministic State Access (IPLD Reachability)

65ebac0

Stebalien mentioned this pull request Jul 28, 2023

feat: implement IPLD reachability analysis filecoin-project/ref-fvm#1824

Merged

Stebalien added 2 commits July 27, 2023 21:17

add discussion url

733d9c0

add wip implementation url

2671af8

ZenGround0 reviewed Jul 28, 2023

View reviewed changes

link to the IPLD memory model

730d0a9

anorth reviewed Jul 30, 2023

View reviewed changes

Stebalien added 2 commits August 1, 2023 18:10

add gas

d011e2a

add an abstract

a28564e

Stebalien marked this pull request as ready for review August 2, 2023 01:50

Stebalien requested review from momack2, arajasek, jennijuju, kaitlin-beegle and raulk as code owners August 2, 2023 01:50

expand on IPLD validation rational

f66dc1a

anorth approved these changes Aug 2, 2023

View reviewed changes

FIPS/fip-XXXX.md Outdated Show resolved Hide resolved

Update FIPS/fip-XXXX.md

5d5bfeb

Co-authored-by: Alex North <[email protected]>

fridrik01 reviewed Aug 10, 2023

View reviewed changes

Stebalien and others added 2 commits August 10, 2023 11:21

Apply suggestions from code review

ba572e4

Co-authored-by: Friðrik Ásmundsson <[email protected]>

address review

5b757a4

kaitlin-beegle reviewed Aug 16, 2023

View reviewed changes

kaitlin-beegle approved these changes Aug 16, 2023

View reviewed changes

fix capitalization

ed64e2d

jennijuju reviewed Aug 22, 2023

View reviewed changes

jennijuju changed the title ~~Deterministic State Access (IPLD Reachability)~~ FIP-0071: Deterministic State Access (IPLD Reachability) Aug 22, 2023

Stebalien added 2 commits August 21, 2023 22:22

review: comment on why we ignore piece CIDs

8df547c

assign FIP number

563ab5b

Stebalien merged commit 87754d8 into filecoin-project:master Aug 22, 2023
1 check passed

Stebalien deleted the steb/ipld-analysis branch August 22, 2023 05:26

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FIP-0071: Deterministic State Access (IPLD Reachability) #763

FIP-0071: Deterministic State Access (IPLD Reachability) #763

Stebalien commented Jul 28, 2023 •

edited

Loading

ZenGround0 Jul 28, 2023

Stebalien Jul 28, 2023

Stebalien Jul 28, 2023

ZenGround0 left a comment

ZenGround0 Jul 28, 2023

Stebalien Jul 28, 2023

Stebalien commented Jul 28, 2023

anorth left a comment

Stebalien commented Aug 2, 2023

fridrik01 left a comment

fridrik01 Jul 31, 2023

Stebalien Aug 10, 2023

fridrik01 Jul 31, 2023

Stebalien Aug 10, 2023

Stebalien Aug 10, 2023

kaitlin-beegle Aug 16, 2023

Stebalien Aug 16, 2023

kaitlin-beegle left a comment

jennijuju Aug 22, 2023

Stebalien Aug 22, 2023

jennijuju Aug 22, 2023

jennijuju commented Aug 22, 2023

Stebalien commented Aug 22, 2023

jennijuju commented Aug 22, 2023


		We will call these "allowed CIDs".

		Additionally, FVM block link analysis will ignore CIDs of sealed and unsealed commitments, as long as the multihash _digest_ is less than 64 bytes:

		@@ -0,0 +1,313 @@
		---
		fip: "<to be assigned>" <!--keep the qoutes around the fip number, i.e: `fip: "0001"`-->

FIP-0071: Deterministic State Access (IPLD Reachability) #763

FIP-0071: Deterministic State Access (IPLD Reachability) #763

Conversation

Stebalien commented Jul 28, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ZenGround0 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Stebalien commented Jul 28, 2023

anorth left a comment

Choose a reason for hiding this comment

Stebalien commented Aug 2, 2023

fridrik01 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kaitlin-beegle left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jennijuju commented Aug 22, 2023

Stebalien commented Aug 22, 2023

jennijuju commented Aug 22, 2023

Stebalien commented Jul 28, 2023 •

edited

Loading