Small chunk heuristics for data density graph

[jprochazk]

What

The data density graph now renders individual events if it is not prohibitively expensive.

• Part of Optimize TimeHistogram #6065

Checklist

• I have read and agree to Contributor Guide and the Code of Conduct
• I've included a screenshot or gif (if applicable)
• I have tested the web demo (if applicable):
  • Using examples from latest main build: rerun.io/viewer
  • Using full set of examples from nightly build: rerun.io/viewer
• The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG
• If applicable, add a new check to the release checklist!
• If have noted any breaking changes to the log API in CHANGELOG.md and the migration guide

• PR Build Summary
• Recent benchmark results
• Wasm size tracking

To run all checks from main, comment on the PR with @rerun-bot full-check.

[jprochazk]

How I read the benchmark results is that it doesn't matter how the chunks are rendered, it scales with how many total events there are. As expected, unsorted data is slower to process. What I don't understand is why splitting chunks seems to do... nothing? The difference between split_never and split_sorted_always is negligible

Benchmark results on my machine:

single_chunks/split_never/0/unsorted               [318.91 ns 320.04 ns 321.80 ns]
single_chunks/split_never/1/unsorted               [606.50 ns 608.50 ns 611.19 ns]
single_chunks/split_never/10/unsorted              [750.01 ns 752.40 ns 755.10 ns]
single_chunks/split_never/100/unsorted             [2.1350 µs 2.1739 µs 2.2272 µs]
single_chunks/split_never/1000/unsorted            [15.506 µs 15.597 µs 15.729 µs]
single_chunks/split_never/10000/unsorted           [157.16 µs 160.55 µs 168.48 µs]
single_chunks/split_never/100000/unsorted          [1.6841 ms 1.7025 ms 1.7256 ms]

single_chunks/split_never/0/sorted                 [299.29 ns 301.88 ns 305.30 ns]
single_chunks/split_never/1/sorted                 [608.50 ns 610.61 ns 612.78 ns]
single_chunks/split_never/10/sorted                [748.52 ns 750.07 ns 753.67 ns]
single_chunks/split_never/100/sorted               [2.1383 µs 2.1671 µs 2.2022 µs]
single_chunks/split_never/1000/sorted              [15.431 µs 16.545 µs 18.757 µs]
single_chunks/split_never/10000/sorted             [144.41 µs 144.85 µs 145.55 µs]
single_chunks/split_never/100000/sorted            [1.4459 ms 1.4634 ms 1.5014 ms]

single_chunks/split_sorted_always/0/unsorted       [324.09 ns 324.52 ns 325.29 ns]
single_chunks/split_sorted_always/1/unsorted       [627.30 ns 630.90 ns 636.81 ns]
single_chunks/split_sorted_always/10/unsorted      [769.69 ns 772.88 ns 775.29 ns]
single_chunks/split_sorted_always/100/unsorted     [2.1289 µs 2.1795 µs 2.2480 µs]
single_chunks/split_sorted_always/1000/unsorted    [15.630 µs 15.700 µs 15.812 µs]
single_chunks/split_sorted_always/10000/unsorted   [157.42 µs 158.44 µs 159.59 µs]
single_chunks/split_sorted_always/100000/unsorted  [1.6889 ms 1.6935 ms 1.6975 ms]

single_chunks/split_sorted_always/0/sorted         [323.92 ns 324.23 ns 324.69 ns]
single_chunks/split_sorted_always/1/sorted         [624.74 ns 627.16 ns 630.03 ns]
single_chunks/split_sorted_always/10/sorted        [768.14 ns 772.79 ns 776.70 ns]
single_chunks/split_sorted_always/100/sorted       [2.1180 µs 2.1324 µs 2.1490 µs]
single_chunks/split_sorted_always/1000/sorted      [15.217 µs 15.278 µs 15.372 µs]
single_chunks/split_sorted_always/10000/sorted     [144.50 µs 160.22 µs 193.26 µs]
single_chunks/split_sorted_always/100000/sorted    [1.4377 ms 1.4402 ms 1.4425 ms]

many_chunks/split_never/100x0                      [317.78 ns 318.48 ns 319.43 ns]
many_chunks/split_never/100x1                      [10.391 µs 10.436 µs 10.480 µs]
many_chunks/split_never/100x10                     [23.736 µs 23.829 µs 23.900 µs]
many_chunks/split_never/100x100                    [159.06 µs 159.49 µs 159.88 µs]
many_chunks/split_never/100x1000                   [1.4715 ms 1.4752 ms 1.4807 ms]

many_chunks/split_sorted_always/100x0              [316.67 ns 317.13 ns 317.59 ns]
many_chunks/split_sorted_always/100x1              [10.333 µs 10.367 µs 10.408 µs]
many_chunks/split_sorted_always/100x10             [23.908 µs 24.016 µs 24.118 µs]
many_chunks/split_sorted_always/100x100            [158.30 µs 158.90 µs 159.40 µs]
many_chunks/split_sorted_always/100x1000           [1.4681 ms 1.4715 ms 1.4742 ms]

[jprochazk]

The code to disable the individual point rendering was wrong, new results make a lot more sense:

• A whole-chunk render has a low fixed cost
• A split-chunk render cost scales by the number of events, but not by the number of chunks

On my machine, building a graph consisting of 100k events costs around 1.5ms, so I think it's a good threshold.

single_chunks/split_never/0/sorted                  [304.53 ns 305.09 ns 306.19 ns]
single_chunks/split_never/1/sorted                  [538.14 ns 539.42 ns 540.75 ns]
single_chunks/split_never/10/sorted                 [7.3167 µs 7.3279 µs 7.3418 µs]
single_chunks/split_never/100/sorted                [8.0007 µs 8.0082 µs 8.0261 µs]
single_chunks/split_never/1000/sorted               [8.0637 µs 8.0697 µs 8.0778 µs]
single_chunks/split_never/10000/sorted              [8.0710 µs 8.0774 µs 8.0872 µs]
single_chunks/split_never/100000/sorted             [7.9158 µs 7.9323 µs 7.9475 µs]

single_chunks/split_never/0/unsorted                [323.15 ns 323.86 ns 324.63 ns]
single_chunks/split_never/1/unsorted                [535.85 ns 538.42 ns 541.81 ns]
single_chunks/split_never/10/unsorted               [7.9926 µs 7.9987 µs 8.0132 µs]
single_chunks/split_never/100/unsorted              [8.0415 µs 8.0488 µs 8.0655 µs]
single_chunks/split_never/1000/unsorted             [8.0766 µs 8.0803 µs 8.0851 µs]
single_chunks/split_never/10000/unsorted            [8.0152 µs 8.0308 µs 8.0459 µs]
single_chunks/split_never/100000/unsorted           [7.9174 µs 7.9384 µs 7.9513 µs]

single_chunks/split_sorted_always/0/sorted          [316.29 ns 316.50 ns 316.81 ns]
single_chunks/split_sorted_always/1/sorted          [628.65 ns 633.78 ns 639.14 ns]
single_chunks/split_sorted_always/10/sorted         [782.03 ns 784.45 ns 786.97 ns]
single_chunks/split_sorted_always/100/sorted        [2.1192 µs 2.1486 µs 2.2146 µs]
single_chunks/split_sorted_always/1000/sorted       [15.142 µs 15.221 µs 15.314 µs]
single_chunks/split_sorted_always/10000/sorted      [144.67 µs 145.13 µs 145.94 µs]
single_chunks/split_sorted_always/100000/sorted     [1.4327 ms 1.4364 ms 1.4432 ms]

single_chunks/split_sorted_always/0/unsorted        [315.55 ns 316.22 ns 317.32 ns]
single_chunks/split_sorted_always/1/unsorted        [627.05 ns 629.86 ns 633.15 ns]
single_chunks/split_sorted_always/10/unsorted       [7.8676 µs 7.9070 µs 7.9602 µs]
single_chunks/split_sorted_always/100/unsorted      [7.9474 µs 7.9530 µs 7.9662 µs]
single_chunks/split_sorted_always/1000/unsorted     [8.0091 µs 8.0272 µs 8.0616 µs]
single_chunks/split_sorted_always/10000/unsorted    [7.9669 µs 7.9766 µs 7.9878 µs]
single_chunks/split_sorted_always/100000/unsorted   [7.9518 µs 7.9639 µs 7.9779 µs]

many_chunks/split_never/100x0                       [317.75 ns 318.01 ns 318.36 ns]
many_chunks/split_never/100x1                       [6.6673 µs 6.6850 µs 6.7080 µs]
many_chunks/split_never/100x10                      [8.0644 µs 8.0746 µs 8.0851 µs]
many_chunks/split_never/100x100                     [11.145 µs 11.233 µs 11.327 µs]
many_chunks/split_never/100x1000                    [14.812 µs 14.855 µs 14.897 µs]

many_chunks/split_sorted_always/100x0               [315.52 ns 315.87 ns 316.55 ns]
many_chunks/split_sorted_always/100x1               [10.280 µs 10.318 µs 10.364 µs]
many_chunks/split_sorted_always/100x10              [23.746 µs 23.850 µs 23.932 µs]
many_chunks/split_sorted_always/100x100             [156.86 µs 158.16 µs 160.45 µs]
many_chunks/split_sorted_always/100x1000            [1.4526 ms 1.4622 ms 1.4816 ms]

[emilk]

I think there is a bug in num_events_cumulative_per_unique_time