Configurable aggregation behaviors for time series plots

[teh-cmc]

Context

With query caching support coming soon ™️, the next bottleneck for displaying many and/or large time series plots is going to be actually rendering them.

While native support for plots in re_renderer seems like the obvious path going forwards in the long term, it'll take a lot of work to get there.
In the meantime, egui_plot is still our best bet.

The plot view currently works by following these rough steps:

1. Query all the necessary data according to the current visible history query.
   Very very costly as of today, but about to be orders of magnitude cheaper with the introduction of query caching.
2. Iterate through the data in order to generate the appropriate egui_plot primitives (points & lines).
   This is generally relatively fast in practice, although it can get very costly for degenerate cases (e.g. all points in the plot have different attributes).
3. Tessellate the egui_plot primitives.
   This happens on the CPU and takes about 3ms for 100k points on my machine IIRC.
4. Rendering.
   Plain old GPU rendering of the generated triangles... but keep in mind: there can be a lot of overdraw!

Proposal

The proposal is to introduce the notion of aggregation functions (MAX, AVG...) to our plot view.
If you squint at it, you can see aggregation functions as a kind of deterministic, user-controlled LOD mechanism.

The idea is straightforward:

1. Introduce a way of asking the plot what's the range of a tick on the X axis at the current zoom level.

• Might or might not exist already, I don't know.
• Because it's all immediate mode, there's the usual chicken and egg problem, but I'm sure we'll find a way.

2. Pre-aggregate the query results based on the range retrieved in step 1, so we get a single value per visible tick.
   We're already iterating through all results anyhow, so this won't add much cost.
3. Only tessellate/render the pre-aggregated results.
   Should bring the rendering costs to near 0.

The aggregation function used would be configurable via a spaceview setting (blueprint!).
We would provide all the usual suspects: NONE, MIN, MAX, AVG, P90, P95, P99, P999.

NONE matches today's behavior: you get a faithful albeit potentially very messy and very slow representation of your data.
Anything else is a tradeoff in accurary in favor of performance/visibility. The hover UI would reflect that.

This feature would still be useful even once we switch to re_renderer-powered plots.

TBD

• Heuristics: it would probably make sense to default the plots to something other than NONE if there more than N plots or P points in the recording.
• How do secondary attributes (color, radius, scattered...) aggregate?
• How does aggregation behave with continuous/decimal tick ranges?

[Wumpf]

great writeup!
I'd love if the answer would be that aggregations are down to be done on the GPU on the fly. But that incurs extra memory "upload" costs, doesn't fix pressure on the queries/query cache and is pretty cumbersome to pull off without compute shaders.