Skip to content

Commit

Permalink
New data APIs 7: RangeZip iterator machinery (#5679)
Browse files Browse the repository at this point in the history
Code generator, and code generated, for the new `RangeZip` machinery.

Similar to #5573, the code generation is implemented as a very low-tech
binary in the crate itself (`src/bin/range_zip.rs`), that just spews the
generated code on stdout.
That seems like the right complexity-to-maintenance tradeoff,
considering that iterator combinators don't really ever change.

Here's an example of one of these combinators:
```rust
/// Returns a new [`RangeZip2x2`] iterator.
///
/// The number of elements in a range zip iterator corresponds to the number of elements in the
/// shortest of its required iterators (`r0`, `r1`).
///
/// Each call to `next` is guaranteed to yield the next value for each required iterator,
/// as well as the most recent index amongst all of them.
///
/// Optional iterators accumulate their state and yield their most recent value (if any),
/// each time the required iterators fire.
pub fn range_zip_2x2<Idx, IR0, R0, IR1, R1, IO0, O0, IO1, O1>(
    r0: IR0,
    r1: IR1,
    o0: IO0,
    o1: IO1,
) -> RangeZip2x2<Idx, IR0::IntoIter, R0, IR1::IntoIter, R1, IO0::IntoIter, O0, IO1::IntoIter, O1>
where
    Idx: std::cmp::Ord,
    IR0: IntoIterator<Item = (Idx, R0)>,
    IR1: IntoIterator<Item = (Idx, R1)>,
    IO0: IntoIterator<Item = (Idx, O0)>,
    IO1: IntoIterator<Item = (Idx, O1)>,
{
    RangeZip2x2 {
        r0: r0.into_iter(),
        r1: r1.into_iter(),
        o0: o0.into_iter().peekable(),
        o1: o1.into_iter().peekable(),

        o0_data_latest: None,
        o1_data_latest: None,
    }
}

/// Implements a range zip iterator combinator with 2 required iterators and 2 optional
/// iterators.
///
/// See [`range_zip_2x2`] for more information.
pub struct RangeZip2x2<Idx, IR0, R0, IR1, R1, IO0, O0, IO1, O1>
where
    Idx: std::cmp::Ord,
    IR0: Iterator<Item = (Idx, R0)>,
    IR1: Iterator<Item = (Idx, R1)>,
    IO0: Iterator<Item = (Idx, O0)>,
    IO1: Iterator<Item = (Idx, O1)>,
{
    r0: IR0,
    r1: IR1,
    o0: Peekable<IO0>,
    o1: Peekable<IO1>,

    o0_data_latest: Option<O0>,
    o1_data_latest: Option<O1>,
}

impl<Idx, IR0, R0, IR1, R1, IO0, O0, IO1, O1> Iterator
    for RangeZip2x2<Idx, IR0, R0, IR1, R1, IO0, O0, IO1, O1>
where
    Idx: std::cmp::Ord,
    IR0: Iterator<Item = (Idx, R0)>,
    IR1: Iterator<Item = (Idx, R1)>,
    IO0: Iterator<Item = (Idx, O0)>,
    IO1: Iterator<Item = (Idx, O1)>,
    O0: Clone,
    O1: Clone,
{
    type Item = (Idx, R0, R1, Option<O0>, Option<O1>);

    #[inline]
    fn next(&mut self) -> Option<Self::Item> {
        let Self {
            r0,
            r1,
            o0,
            o1,
            o0_data_latest,
            o1_data_latest,
        } = self;

        let Some((r0_index, r0_data)) = r0.next() else {
            return None;
        };
        let Some((r1_index, r1_data)) = r1.next() else {
            return None;
        };

        let max_index = [r0_index, r1_index].into_iter().max().unwrap();

        let mut o0_data = None;
        while let Some((_, data)) = o0.next_if(|(index, _)| index <= &max_index) {
            o0_data = Some(data);
        }
        let o0_data = o0_data.or(o0_data_latest.take());
        *o0_data_latest = o0_data.clone();

        let mut o1_data = None;
        while let Some((_, data)) = o1.next_if(|(index, _)| index <= &max_index) {
            o1_data = Some(data);
        }
        let o1_data = o1_data.or(o1_data_latest.take());
        *o1_data_latest = o1_data.clone();

        Some((max_index, r0_data, r1_data, o0_data, o1_data))
    }
}
```

---

Part of a PR series to completely revamp the data APIs in preparation
for the removal of instance keys and the introduction of promises:
- #5573
- #5574
- #5581
- #5605
- #5606
- #5633
- #5673
- #5679
- #5687
- #5755
- TODO
- TODO

Builds on top of the static data PR series:
- #5534
  • Loading branch information
teh-cmc authored Apr 8, 2024
1 parent c53d339 commit d4c6469
Show file tree
Hide file tree
Showing 6 changed files with 4,125 additions and 1 deletion.
5 changes: 5 additions & 0 deletions crates/re_query2/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,11 @@ name = "clamped_zip"
required-features = ["codegen"]


[[bin]]
name = "range_zip"
required-features = ["codegen"]


[[bench]]
name = "latest_at"
harness = false
Loading

0 comments on commit d4c6469

Please sign in to comment.