Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Low-level logging APIs cannot splat #4742

Closed
teh-cmc opened this issue Jan 8, 2024 · 1 comment · Fixed by #5573
Closed

Low-level logging APIs cannot splat #4742

teh-cmc opened this issue Jan 8, 2024 · 1 comment · Fixed by #5573
Assignees
@teh-cmc
Labels
🪳 bug Something isn't working 🪵 Log & send APIs Affects the user-facing API for all languages

Comments

@teh-cmc
Copy link
Member

teh-cmc commented Jan 8, 2024

The following will not do what you'd expect, since it doesn't know that the main component has multiple instances:

import rerun as rr

rr.init("rerun_example_points3d_simple", spawn=True)

rr.set_time_sequence("frame", 1)
rr.log("points", rr.Points3D([[0, 0, 0], [1, 1, 1]]))

# This will only color `point[0]`.
rr.set_time_sequence("frame", 2)
rr.log_components("points", [rr.components.ColorBatch([0xFF0000FF])])

Worse, there's no public way to explicitly indicate that we want it to splat.

The only way is to go through multiple layers of private APIs:

rr.log_components("points", [rr.components.ColorBatch([0xFF0000FF]), rr._log._splat()])

and even then, the above does not work, you need to recast it some more for some reason:

rr.log_components("points", [rr.components.ColorBatch([0xFF0000FF]), rr.components.InstanceKeyBatch(rr._log._splat())])

that works.

This would be solved by the "clamp-to-last" data model that we've talked about in Stockholm (for which I still need to write an issue at some point...).
@teh-cmc teh-cmc added 🪳 bug Something isn't working 🪵 Log & send APIs Affects the user-facing API for all languages labels Jan 8, 2024
@jleibs
Copy link
Member

jleibs commented Jan 8, 2024

You can also do:

rr.log_components("points", [rr.components.ColorBatch([0xFF0000FF])], num_instances=2)

@teh-cmc teh-cmc self-assigned this Mar 15, 2024
@teh-cmc teh-cmc mentioned this issue Mar 19, 2024
Merged
5 tasks
@teh-cmc teh-cmc linked a pull request Mar 19, 2024 that will close this issue
New data APIs 0: ClampedZip iterator machinery #5573
Merged
5 tasks
teh-cmc added a commit that referenced this issue Apr 8, 2024
@teh-cmc
New data APIs 0: ClampedZip iterator machinery (#5573)
e158114 
This introduces a new temporary `re_query2` crate, which won't ever be
published.
It will replace the existing `re_query` crate once all the necessary
features have been backported.

As of this PR, this crate only contains the `ClampedZip` iterator
machinery, which is code generated for all the different arities.

Since I'm very, _very tired_ of the awful DX of macros, I implemented a
very low-tech code generator in the crate itself
(`src/bin/clamped_zip.rs`) that just spews the generated code on stdout.
That seems like the right complexity-to-maintenance tradeoff,
considering that iterator combinators don't really ever change.

`ClampedZip` naturally works with more than one required component,
finally!

- Fixes #4742
- Fixes #2750

Here's an example of one of these combinators:
```rust
/// Returns a new [`ClampedZip1x2`] iterator.
///
/// The number of elements in a clamped zip iterator corresponds to the number of elements in the
/// shortest of its required iterators (`r0`).
///
/// Optional iterators (`o0`, `o1`) will repeat their latest values if they happen to be to short
/// to be zipped with the shortest of the required iterators.
///
/// If an optional iterator is not only too short but actually empty, its associated default function
/// (`o0_default_fn`, `o1_default_fn`) will be executed and the resulting value repeated as necessary.
pub fn clamped_zip_1x2<R0, O0, O1, D0, D1>(
    r0: R0,
    o0: O0,
    o0_default_fn: D0,
    o1: O1,
    o1_default_fn: D1,
) -> ClampedZip1x2<R0::IntoIter, O0::IntoIter, O1::IntoIter, D0, D1>
where
    R0: IntoIterator,
    O0: IntoIterator,
    O0::Item: Clone,
    O1: IntoIterator,
    O1::Item: Clone,
    D0: Fn() -> O0::Item,
    D1: Fn() -> O1::Item,
{
    ClampedZip1x2 {
        r0: r0.into_iter(),
        o0: o0.into_iter(),
        o1: o1.into_iter(),
        o0_default_fn,
        o1_default_fn,
        o0_latest_value: None,
        o1_latest_value: None,
    }
}

/// Implements a clamped zip iterator combinator with 2 required iterators and 2 optional
/// iterators.
///
/// See [`clamped_zip_1x2`] for more information.
pub struct ClampedZip1x2<R0, O0, O1, D0, D1>
where
    R0: Iterator,
    O0: Iterator,
    O0::Item: Clone,
    O1: Iterator,
    O1::Item: Clone,
    D0: Fn() -> O0::Item,
    D1: Fn() -> O1::Item,
{
    r0: R0,
    o0: O0,
    o1: O1,
    o0_default_fn: D0,
    o1_default_fn: D1,

    o0_latest_value: Option<O0::Item>,
    o1_latest_value: Option<O1::Item>,
}

impl<R0, O0, O1, D0, D1> Iterator for ClampedZip1x2<R0, O0, O1, D0, D1>
where
    R0: Iterator,
    O0: Iterator,
    O0::Item: Clone,
    O1: Iterator,
    O1::Item: Clone,
    D0: Fn() -> O0::Item,
    D1: Fn() -> O1::Item,
{
    type Item = (R0::Item, O0::Item, O1::Item);

    #[inline]
    fn next(&mut self) -> Option<Self::Item> {
        let r0_next = self.r0.next()?;
        let o0_next = self.o0.next().or(self.o0_latest_value.take());
        let o1_next = self.o1.next().or(self.o1_latest_value.take());

        self.o0_latest_value = o0_next.clone();
        self.o1_latest_value = o1_next.clone();

        Some((
            r0_next,
            o0_next.unwrap_or_else(|| (self.o0_default_fn)()),
            o1_next.unwrap_or_else(|| (self.o1_default_fn)()),
        ))
    }
}
```

---

Part of a PR series to completely revamp the data APIs in preparation
for the removal of instance keys and the introduction of promises:
- #5573
- #5574
- #5581
- #5605
- #5606
- #5633
- #5673
- #5679
- #5687
- #5755
- TODO
- TODO

Builds on top of the static data PR series:
- #5534
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@teh-cmc teh-cmc

Labels
🪳 bug Something isn't working 🪵 Log & send APIs Affects the user-facing API for all languages
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

New data APIs 0: ClampedZip iterator machinery rerun-io/rerun
2 participants
@teh-cmc @jleibs