Skip to content

Commit

Permalink
dont split on index bucket size (#1558)
Browse files Browse the repository at this point in the history
  • Loading branch information
@teh-cmc
teh-cmc authored Mar 10, 2023
1 parent 2977a5f commit fbcf009
Showing 1 changed file with 9 additions and 1 deletion.
10 changes: 9 additions & 1 deletion crates/re_data_store/src/log_db.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,15 @@ impl Default for EntityDb {
InstanceKey::name(),
DataStoreConfig {
component_bucket_size_bytes: 1024 * 1024, // 1 MiB
index_bucket_size_bytes: 1024, // 1KiB
// We do not garbage collect index buckets at the moment, and so the size of
// individual index buckets is irrelevant, only their total number of rows
// matter.
// See <pr-link> for details.
//
// TODO(cmc): Bring back index GC once the whole `MsgId` mismatch issue is
// resolved (probably once batching is implemented).
index_bucket_size_bytes: u64::MAX,
index_bucket_nb_rows: 2048,
..Default::default()
},
),
Expand Down

4 comments on commit fbcf009

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rust Benchmark

Benchmark suite Current: fbcf009 Previous: 2977a5f Ratio
datastore/insert/batch/rects/insert 564196 ns/iter (± 3391) 570245 ns/iter (± 2018) 0.99
datastore/latest_at/batch/rects/query 1858 ns/iter (± 8) 1853 ns/iter (± 130) 1.00
datastore/latest_at/missing_components/primary 286 ns/iter (± 0) 286 ns/iter (± 0) 1
datastore/latest_at/missing_components/secondaries 431 ns/iter (± 0) 431 ns/iter (± 0) 1
datastore/range/batch/rects/query 150874 ns/iter (± 1113) 149865 ns/iter (± 336) 1.01
mono_points_arrow/generate_message_bundles 65487073 ns/iter (± 863842) 44586319 ns/iter (± 1106633) 1.47
mono_points_arrow/generate_messages 173226692 ns/iter (± 2083871) 125462521 ns/iter (± 1056426) 1.38
mono_points_arrow/encode_log_msg 208232306 ns/iter (± 2299725) 152573056 ns/iter (± 699404) 1.36
mono_points_arrow/encode_total 446657398 ns/iter (± 2849127) 324536184 ns/iter (± 1578215) 1.38
mono_points_arrow/decode_log_msg 216386769 ns/iter (± 1566828) 177599408 ns/iter (± 837755) 1.22
mono_points_arrow/decode_message_bundles 104802303 ns/iter (± 1375784) 63512761 ns/iter (± 1095669) 1.65
mono_points_arrow/decode_total 315038842 ns/iter (± 2401113) 244545303 ns/iter (± 1944702) 1.29
batch_points_arrow/generate_message_bundles 334384 ns/iter (± 930) 331965 ns/iter (± 555) 1.01
batch_points_arrow/generate_messages 6291 ns/iter (± 19) 6272 ns/iter (± 32) 1.00
batch_points_arrow/encode_log_msg 379301 ns/iter (± 3382) 366736 ns/iter (± 1010) 1.03
batch_points_arrow/encode_total 734920 ns/iter (± 4509) 722095 ns/iter (± 1330) 1.02
batch_points_arrow/decode_log_msg 361831 ns/iter (± 3698) 350168 ns/iter (± 766) 1.03
batch_points_arrow/decode_message_bundles 2128 ns/iter (± 8) 2112 ns/iter (± 4) 1.01
batch_points_arrow/decode_total 362318 ns/iter (± 4466) 357365 ns/iter (± 952) 1.01
arrow_mono_points/insert 10057759421 ns/iter (± 117538766) 6144606077 ns/iter (± 10248151) 1.64
arrow_mono_points/query 1839928 ns/iter (± 33863) 1730367 ns/iter (± 5307) 1.06
arrow_batch_points/insert 2725546 ns/iter (± 111980) 2704817 ns/iter (± 22176) 1.01
arrow_batch_points/query 16948 ns/iter (± 19) 16825 ns/iter (± 17) 1.01
arrow_batch_vecs/insert 43307 ns/iter (± 174) 42494 ns/iter (± 197) 1.02
arrow_batch_vecs/query 507986 ns/iter (± 1009) 508134 ns/iter (± 1346) 1.00
tuid/Tuid::random 34 ns/iter (± 0) 34 ns/iter (± 0) 1

This comment was automatically generated by workflow using github-action-benchmark.

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark 'Rust Benchmark'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.50.

Benchmark suite Current: fbcf009 Previous: 2977a5f Ratio
mono_points_arrow/decode_message_bundles 104802303 ns/iter (± 1375784) 63512761 ns/iter (± 1095669) 1.65
arrow_mono_points/insert 10057759421 ns/iter (± 117538766) 6144606077 ns/iter (± 10248151) 1.64

This comment was automatically generated by workflow using github-action-benchmark.

@teh-cmc
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't get how this patch could impact any of these two benchmarks?

@teh-cmc
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to have gone back to normal, just a broken CI runner.

Please sign in to comment.