fastpath CBOR #64
fastpath CBOR #64
Conversation
|
Thoughts here? especially @whyrusleeping |
|
Hiyo! Does this have any chance of getting merged? Looking for feedback here. |
| var err error | ||
| selfMarshaling, ok := obj.(cborMarshaler) | ||
| if ok { | ||
| err = selfMarshaling.MarshalCBOR(w) |
There was a problem hiding this comment.
+1 this is what i had in mind
| m.reader.r = r | ||
| err := m.unmarshal.Unmarshal(obj) | ||
| selfUnmarshaler, ok := obj.(cborUnmarshaler) |
node.go
Outdated
| @@ -261,6 +274,10 @@ func (n *Node) ResolveLink(path []string) (*node.Link, []string, error) { | |||
|
|
|||
| // Tree returns a flattend array of paths at the given path for the given depth. | |||
| func (n *Node) Tree(path string, depth int) []string { | |||
| err := n.setTreeAndLinks() | |||
| if err != nil { | |||
There was a problem hiding this comment.
quietly dropping the error here makes me uncomfortable
There was a problem hiding this comment.
Agree, but I didn't want to change the interface.
There was a problem hiding this comment.
The marshaling and cloner changes LGTM. Happy to ship those right now.
The changes to the tree and links thing is another story though. Definitely not comfortable ignoring errors.
I've also been thinking about how to improve this separately, and I think that writing a piece of code that just iterates through the raw cbor directly to pull out the links and tree. This can be way faster than the current way we do it (should be as fast, if not faster than, the parsing itself, and could potentially be combined with the parsing in a future optimization). If that sounds good to you, i'd say we pull those changes out of this PR for now, get this merged, and then do that as a followup.
|
(also, sorry for the slowness in response, been not checking my email lately) |
tobowers
force-pushed
the
feature/fastpath
branch
from
5fcb9a4
to
4f57472
Compare
tobowers
force-pushed
the
feature/fastpath
branch
from
4f57472
to
9847526
Compare
|
ok, I made the incremental change to support objects which can encode/decode themselves. That being said, this repo won't see any speed improvement due to these changes until we can fix the decode into interface{} |
|
Oh... I guess I'm wrong... only projects using merkledag.Decode will still have the slow path... but general BlockService and Blockstore will not actually go through that method. |
There was a problem hiding this comment.
Not one of designated reviewers, but this now looks LGTM to me. @warpfork / @whyrusleeping ?
I realize this might not be mergable as is... because it changes how Tree() and Links() are handled and doesn't error in the same place as before.
I was looking at this PR: https://github.com/ipfs/go-hamt-ipld/pull/29/files and looking at some benchmarks noticed that refmt was still taking up a bunch of time in our blockservice interface and I realized that's because when the block comes back it is still decoded into an interface{}.
This PR allows lazy loading of Tree and Links (which are often unused) and allows fast-path cbor if objects support them.