-
Notifications
You must be signed in to change notification settings - Fork 50
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
A more Featureful Approach to Storage APIs #265
Commits on Oct 10, 2021
-
storage: start new API based on feature detection.
This will eventually rival and replace linking.BlockWriteOpener and linking.BlockReadOpener, being more primitive than them, and easier to implement without importing any go-ipld-prime types. (The eventual migration story should be smooth, though; we'll probably support both styles at once for a while.) It also explicitly does not acknolwedge the interfaces from the github.com/ipfs/go-datastore module. We're seeking several key improvements over those, including: - slimmer core; more feature-detection - only standard library types needed to implement the API - context everywhere - clear support for streaming operation - separation between read and write directions of operation - a consistent strategy for feature-detection and optional fastpaths - key type should be much simpler (and not do strange internal manipulations that cause useless allocations!) We'll offer wrappers that bridge back to the go-datastore style API, because there's lots of things implemented today using it. However, these objectives for a new clean API are not reasonably feasible to reach by incrementally evolving that older interface, so we have to start anew. In this commit: we have a first draft of the most essential interfaces, as well as always-available functions at package scope which will "do the right thing" in the most efficient way possible, by using feature detection internally on your behalf, for ease-of-use.
Configuration menu - View commit details
-
Copy full SHA for 85fcf24 - Browse repository at this point
Copy the full SHA 85fcf24View commit details -
add binary access to the interface for Link.
This is a pretty big change if anyone else out there is implementing new link types. If you're using the linking/cid package, the change is already done. I'm going to consider this a minimally breaking change because I don't think we've got a lot of diverse Link implementations out in the wild. (And even if so: this should be a pretty easy addition to make.) I'm anticipating using this as part of a good layering of APIs regarding storage. (We should be able to use binary, dense things in any storage substrate that allows it. Some common storage systems, like e.g. flatfs, don't work with binary. But any escaping having to do with that should be associated with the storage system -- not pushed to any other layer. Having binary access here helps do that.)
Configuration menu - View commit details
-
Copy full SHA for 3b4df1e - Browse repository at this point
Copy the full SHA 3b4df1eView commit details -
Fixes import cycles that I would otherwise be about to experience starting in the next commit.
Configuration menu - View commit details
-
Copy full SHA for 0fdf939 - Browse repository at this point
Copy the full SHA 0fdf939View commit details -
linking: new method to help configure LinkSystem to use interfaces de…
…fined by the storage package. This means we now have a one-liner call that can: - rig up a storage system... - which can have been written while using zero types from this repo... - and it'll DTRT. - All while it's also become much easier to write storage implementations, as well as extend them gracefully, because of the new storage package APIs. This seems good.
Configuration menu - View commit details
-
Copy full SHA for 5711767 - Browse repository at this point
Copy the full SHA 5711767View commit details -
storage/dsadapter: a small module which bridges go-datastore forward …
…into the new storage API.
Configuration menu - View commit details
-
Copy full SHA for 522500c - Browse repository at this point
Copy the full SHA 522500cView commit details
Commits on Oct 14, 2021
-
storage: Peek API talks about close explicitly.
Opens the way to a storage implementation that makes reuse of buffers.
Configuration menu - View commit details
-
Copy full SHA for c56158d - Browse repository at this point
Copy the full SHA c56158dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 55a4896 - Browse repository at this point
Copy the full SHA 55a4896View commit details -
storage/memstore: update to match the new interfaces.
Most usages change from a one-liner to another one-liner. Some tests get a bit more involved, because they were actually using the callback structure to hook introspections up.
Configuration menu - View commit details
-
Copy full SHA for 273362c - Browse repository at this point
Copy the full SHA 273362cView commit details