Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature requests: Change Streams and File I/O Abstraction #1

Open
Type1J opened this issue Dec 12, 2021 · 2 comments
Open

Feature requests: Change Streams and File I/O Abstraction #1

Type1J opened this issue Dec 12, 2021 · 2 comments

Comments

@Type1J
Copy link

Type1J commented Dec 12, 2021

Hello, I didn't see a better way to create a feature request, so I created this issue. I have 2 feature requests:

  1. Change Streams: Active queries, or queries that report new items (sorting can be disallowed on these queries) if the database adds, updates (old and new reported), or removes something that would have been included in a result set. This could also, just be a notification (callback or Stream entry) that the result set might have changed.
  2. File I/O should be a trait that is optionally passed in when opening the database. This allows storage in environments where storage is available, but the std library isn't implemented (embedded systems, wasm, etc.), and it could also be used to allow encryption to be added external to the database library.
@dyedgreen
Copy link
Owner

  1. Sounds interesting!

  2. I was previously thinking of having a networked pager, which shares transactions with a cluster of other deployments peer-to-peer (imagine running multiple servers and having the database sync between them automatically).

    I'm not sure I want to go down this road, but it might be a pretty useful feature. But if so, that might not work very well with alternate storage. Currently, this uses sanakirja as a pager and b-tree implementation. So supporting this would probably require either contributing this feature to sanakirja, or writing an alternative pager.

@Type1J
Copy link
Author

Type1J commented Dec 15, 2021

For 2, let me be more specific in what I'd like to do.

Have you seen PouchDB? It allows one to store data using IndexedDB in a browser while providing a CouchDB-like interface. It also syncs data from a remote CouchDB just as CouchDB would replicate with another CouchDB.

I was hoping to provide a query on the server that publishes data to be synced with a subscriber on the client. If the mechanism is flexible enough to add to other databases, then I was going to use RedisGraph on the server side and cqlite on the client, but if your cluster idea is fast and reliable, then I may just use cqlite on both sides.

On the client, I'd like to be updated as data changes on the server (probably using either a gRPC-web stream, websocket, or server sent events connection), and when that change is added to the in-browser database, I wanted a notification, so I can redraw the UI.

I do something like this now with PouchDB, but I'd really like a graph database using Cypher on both sides. My main site logic is in Rust WASM, so cqlite would integrate well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants