Skip to content

Commit

Permalink
RFC: datastore state of the union & end-to-end batching (#1610)
Browse files Browse the repository at this point in the history 
* add batching rfc

* Update design/batching.md

Co-authored-by: Emil Ernerfeldt <[email protected]>

* Update design/batching.md

Co-authored-by: Emil Ernerfeldt <[email protected]>

* Update design/batching.md

Co-authored-by: Emil Ernerfeldt <[email protected]>

* Update design/batching.md

Co-authored-by: Emil Ernerfeldt <[email protected]>

* qa component instances vs. instance keys

* no sticky

* qa are there any special components & non-integer instance keys

* give some clue about cell incompatibility

* temporary constructs

* zstd is already setup

* more planning

* date and links

---------

Co-authored-by: Emil Ernerfeldt <[email protected]>
  • Loading branch information
@teh-cmc @emilk
teh-cmc and emilk authored Apr 13, 2023
1 parent 925f531 commit cb95f5e
Showing 1 changed file with 1,022 additions and 0 deletions.
Loading

1 comment on commit cb95f5e

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rust Benchmark

Benchmark suite Current: cb95f5e Previous: 1e2515c Ratio
datastore/num_rows=1000/num_instances=1000/packed=false/insert/default 2800753 ns/iter (± 17865) 2798053 ns/iter (± 49459) 1.00
datastore/num_rows=1000/num_instances=1000/packed=false/latest_at/default 371 ns/iter (± 1) 371 ns/iter (± 2) 1
datastore/num_rows=1000/num_instances=1000/packed=false/latest_at_missing/primary/default 274 ns/iter (± 1) 263 ns/iter (± 1) 1.04
datastore/num_rows=1000/num_instances=1000/packed=false/latest_at_missing/secondaries/default 422 ns/iter (± 17) 419 ns/iter (± 4) 1.01
datastore/num_rows=1000/num_instances=1000/packed=false/range/default 2949675 ns/iter (± 37931) 2955186 ns/iter (± 42069) 1.00
datastore/num_rows=1000/num_instances=1000/gc/default 2366760 ns/iter (± 11699) 2360531 ns/iter (± 7742) 1.00
mono_points_arrow/generate_message_bundles 25690583 ns/iter (± 1360299) 28137558 ns/iter (± 1145599) 0.91
mono_points_arrow/generate_messages 111183648 ns/iter (± 879567) 123496964 ns/iter (± 1152016) 0.90
mono_points_arrow/encode_log_msg 140163972 ns/iter (± 609784) 155039628 ns/iter (± 1412968) 0.90
mono_points_arrow/encode_total 280532139 ns/iter (± 1640498) 310448162 ns/iter (± 2250264) 0.90
mono_points_arrow/decode_log_msg 177118254 ns/iter (± 1235545) 189541902 ns/iter (± 1918086) 0.93
mono_points_arrow/decode_message_bundles 54411011 ns/iter (± 1216228) 67210424 ns/iter (± 792212) 0.81
mono_points_arrow/decode_total 233024184 ns/iter (± 1631503) 253322604 ns/iter (± 1974248) 0.92
mono_points_arrow_batched/generate_message_bundles 18224506 ns/iter (± 293461) 18836640 ns/iter (± 1976435) 0.97
mono_points_arrow_batched/generate_messages 4082562 ns/iter (± 139756) 4104733 ns/iter (± 141087) 0.99
mono_points_arrow_batched/encode_log_msg 1388313 ns/iter (± 3959) 1348043 ns/iter (± 6276) 1.03
mono_points_arrow_batched/encode_total 24871569 ns/iter (± 776979) 28143662 ns/iter (± 1433702) 0.88
mono_points_arrow_batched/decode_log_msg 779621 ns/iter (± 2203) 778172 ns/iter (± 4235) 1.00
mono_points_arrow_batched/decode_message_bundles 7545789 ns/iter (± 51407) 7562054 ns/iter (± 97282) 1.00
mono_points_arrow_batched/decode_total 8368617 ns/iter (± 56581) 9586152 ns/iter (± 711385) 0.87
batch_points_arrow/generate_message_bundles 237398 ns/iter (± 1455) 239887 ns/iter (± 353) 0.99
batch_points_arrow/generate_messages 5029 ns/iter (± 34) 5050 ns/iter (± 18) 1.00
batch_points_arrow/encode_log_msg 255540 ns/iter (± 1386) 265905 ns/iter (± 2538) 0.96
batch_points_arrow/encode_total 530531 ns/iter (± 3390) 528194 ns/iter (± 1674) 1.00
batch_points_arrow/decode_log_msg 208755 ns/iter (± 1143) 213243 ns/iter (± 2000) 0.98
batch_points_arrow/decode_message_bundles 1817 ns/iter (± 14) 1831 ns/iter (± 19) 0.99
batch_points_arrow/decode_total 216531 ns/iter (± 1230) 215967 ns/iter (± 2134) 1.00
arrow_mono_points/insert 2315994449 ns/iter (± 3540663) 2473011849 ns/iter (± 57619355) 0.94
arrow_mono_points/query 1174175 ns/iter (± 9928) 1218937 ns/iter (± 16820) 0.96
arrow_batch_points/insert 1140866 ns/iter (± 6596) 1141669 ns/iter (± 14681) 1.00
arrow_batch_points/query 14673 ns/iter (± 120) 14152 ns/iter (± 87) 1.04
arrow_batch_vecs/insert 26138 ns/iter (± 141) 26391 ns/iter (± 896) 0.99
arrow_batch_vecs/query 323075 ns/iter (± 2524) 321244 ns/iter (± 4311) 1.01
tuid/Tuid::random 34 ns/iter (± 0) 34 ns/iter (± 0) 1

This comment was automatically generated by workflow using github-action-benchmark.

Please sign in to comment.