Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
New data APIs 2: cached latest-at queries (#5581)
Static-aware, key-less, component-based, cached latest-at APIs. The overall structure of this new cache is very similar to what we had before. Effectively it is just an extremely simplified version of `re_query_cache`. This introduces a new temporary `re_query_cache2` crate, which won't ever be published. It will replace the existing `re_query_cache` crate once all the necessary features have been backported. - Fixes #3232 - Fixes #4733 - Fixes #4734 - Part of #3379 - Part of #1893 Example: ```rust let caches = re_query_cache2::Caches::new(&store); // First, get the results for this query. // // They might or might not already be cached. We won't know for sure until we try to access // each individual component's data below. let results: CachedLatestAtResults = caches.latest_at( &store, &query, &entity_path.into(), MyPoints::all_components().iter().cloned(), // no generics! ); // Then, grab the results for each individual components. // * `get_required` returns an error if the component batch is missing // * `get_optional` returns an empty set of results if the component if missing // * `get` returns an option // // At this point we still don't know whether they are cached or not. That's the next step. let points: &CachedLatestAtComponentResults = results.get_required::<MyPoint>()?; let colors: &CachedLatestAtComponentResults = results.get_optional::<MyColor>(); let labels: &CachedLatestAtComponentResults = results.get_optional::<MyLabel>(); // Then comes the time to resolve/convert and deserialize the data. // These steps have to be done together for efficiency reasons. // // Both the resolution and deserialization steps might fail, which is why this returns a `Result<Result<T>>`. // Use `PromiseResult::flatten` to simplify it down to a single result. // // A choice now has to be made regarding the nullability of the _component batch's instances_. // Our IDL doesn't support nullable instances at the moment -- so for the foreseeable future you probably // shouldn't be using anything but `iter_dense`. // // This is the step at which caching comes into play. // // If the data has already been accessed with the same nullability characteristics in the // past, then this will just grab the pre-deserialized, pre-resolved/pre-converted result from // the cache. // // Otherwise, this will trigger a deserialization and cache the result for next time. let points = match points.iter_dense::<MyPoint>(&mut resolver).flatten() { PromiseResult::Pending => { // Handle the fact that the data isn't ready appropriately. return Ok(()); } PromiseResult::Ready(data) => data, PromiseResult::Error(err) => return Err(err.into()), }; let colors = match colors.iter_dense::<MyColor>(&mut resolver).flatten() { PromiseResult::Pending => { // Handle the fact that the data isn't ready appropriately. return Ok(()); } PromiseResult::Ready(data) => data, PromiseResult::Error(err) => return Err(err.into()), }; let labels = match labels.iter_sparse::<MyLabel>(&mut resolver).flatten() { PromiseResult::Pending => { // Handle the fact that the data isn't ready appropriately. return Ok(()); } PromiseResult::Ready(data) => data, PromiseResult::Error(err) => return Err(err.into()), }; // With the data now fully resolved/converted and deserialized, the joining logic can be // applied. // // In most cases this will be either a clamped zip, or no joining at all. let color_default_fn = || { static DEFAULT: MyColor = MyColor(0xFF00FFFF); &DEFAULT }; let label_default_fn = || None; let results = clamped_zip_1x2(points, colors, color_default_fn, labels, label_default_fn).collect_vec(); ``` --- Part of a PR series to completely revamp the data APIs in preparation for the removal of instance keys and the introduction of promises: - #5573 - #5574 - #5581 - #5605 - #5606 - #5633 - #5673 - #5679 - #5687 - #5755 - TODO - TODO Builds on top of the static data PR series: - #5534
- Loading branch information
