sdk/rust: escape hatch for cacheless mode

[vis-eyth]

Description of Changes

These changes allow SDK consumers to register a tokio::sync::mpsc::UnboundedSender<DbUpdate> for a DbConnection that directly receives all database updates instead of the ClientCache.

Rationale

For writing tooling the current client cache feels very inefficient - these changes allowed us to reduce memory consumption by around a factor of 8, from being able to run the tooling for one region of BitCraft Online to all of them. Batch processing all rows in a database update as well as processing them in a different thread also lead to increased performance.

Expected complexity level and risk

1 - for complexity i believe this is a fairly minor change, although not completely aware of your scale.
? - for risk.

Behavioral changes only exist if the channel is registered via DbConnectionBuilder::with_update_channel, currently hidden from the docs. Risk may increase significantly if DbUpdate/TableUpdate channels are not meant to be public API and are in flux.

Testing

I do not have any tests for these changes, but have run a patched SDK with these changes for a while in bitcraft tooling such as https://github.com/vis-eyth/bitcraft-nodeindex.

[CLAassistant]

All committers have signed the CLA.

[gefjon]

You've got two distinct features here, and I think we should treat their interfaces separately. They are:

1. Access to the entire state delta of a transaction in one place, as an alternative to callback routing.
2. Disabling the client cache.

I think we are unlikely to ever make 2 a stable, documented feature of any SpacetimeDB client SDK. In addition to being how users look up rows, maintaining the client cache is necessary to provide some semantic filters on raw updates: a raw TableUpdate may contain inserts for rows already present in the client cache, multiple inserts for the same row, or "phantom" deletes for rows that will remain present in the client cache afterwards. Without the client cache, these oddities will be exposed to users, which seems undesirable and footgun-ey.

That said, I am not opposed to merging 2 as exposed via an extension trait on DbConnectionBuilder from the spacetimedb_sdk::unstable module. This trait should be named WithoutClientCache, and it should expose a method DbConnectionBuilder::without_client_cache(self) -> Self.

1 is a desirable feature which I would be happy to eventually stabilize and port to the other SDKs. However, I'd request two changes:

• Rather than DbUpdate and TableUpdate, clients should be exposed to something akin to a TableAppliedDiff (which is defined in SpacetimeDB/sdks/rust/src/client_cache.rs). When the client cache is enabled, this will hide the internal weirdness of raw updates I mentioned above. It will also identify updates by primary key.
• Rather than supplying a channel when building the connection, clients should register a callback with the DbConnection, which I'd call on_event. This callback should be of type FnMut(&EventContext, DbAppliedDiff), where DbAppliedDiff is a collection of things-like-TableAppliedDiff.

This will require some modifications to the codegen to define the DbAppliedDiff type. We'll also probably end up doing some bikeshedding about the name of those types; we may end up using DbUpdate and TableUpdate for the user-facing types and rename the existing types, or something.