(chore): update benchmarks

[ilan-gold]

This is very preliminary and partially blocked by zarr-developers/zarr-python#2971

In any case, on the machine I am using, zarrs-python sharding performance in dask seems quite degraded, so wanted to raise that to see if you could reproduce.

Some relevatn system info potentially:

System:
  Kernel: 5.15.0-134-generic x86_64 bits: 64 compiler: gcc v: 11.4.0 Console: pty pts/1
    Distro: Ubuntu 22.04.5 LTS (Jammy Jellyfish)
Machine:
  Type: Kvm System: Red Hat product: OpenStack Compute v: 23.2.3-17.1.20240919170758.el9ost
    serial: <superuser required>
  Mobo: Red Hat model: RHEL v: RHEL-9.2.0 PC (Q35 + ICH9, 2009) serial: <superuser required>
    BIOS: SeaBIOS v: 1.16.1-1.el9 date: 04/01/2014
CPU:
  Info: 16x 1-core model: AMD EPYC-Rome bits: 64 type: SMP arch: Zen 2 rev: 0 cache:
    L1: 16x 64 KiB (1024 KiB) L2: 16x 512 KiB (8 MiB) L3: 16x 16 MiB (256 MiB)
  Speed (MHz): avg: 2994 min/max: N/A cores: 1: 2994 2: 2994 3: 2994 4: 2994 5: 2994 6: 2994
    7: 2994 8: 2994 9: 2994 10: 2994 11: 2994 12: 2994 13: 2994 14: 2994 15: 2994 16: 2994
    bogomips: 95819
  Flags: avx avx2 lm nx pae sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3

38: None 00.0: 10600 Disk                                       
  [Created at block.245]
  Unique ID: ndrI.Fxp0d3BezAE
  Parent ID: xXhN.+FFPFBVXZu6
  SysFS ID: /class/block/vdb
  SysFS BusID: virtio3
  SysFS Device Link: /devices/pci0000:00/0000:00:02.3/0000:05:00.0/virtio3
  Hardware Class: disk
  Model: "Disk"
  Driver: "virtio-pci", "virtio_blk"
  Driver Modules: "virtio_pci", "virtio_blk"
  Device File: /dev/vdb
  Device Files: /dev/vdb, /dev/disk/by-path/pci-0000:05:00.0, /dev/disk/by-uuid/cc55ccc8-1df0-402c-8c62-6b9866f78612, /dev/disk/by-path/virtio-pci-0000:05:00.0, /dev/disk/by-id/virtio-31999f45-82e0-4e8d-9
  Device Number: block 252:16-252:31
  Drive status: no medium
  Config Status: cfg=new, avail=yes, need=no, active=unknown
  Attached to: #37 (Storage controller)

This disk is a mounted volume from denbi: https://cloud.denbi.de/wiki/quickstart/#using-cinder-volumes - not sure how much more nfo I can find

[LDeakin]

I'd guess dask is now chunking based on the inner chunk size and previously it was the shard size. So the task overhead would be insane for that dataset which has tiny $32^3$ inner chunks. zarrs/zarrs-python#90 probably does not help too.

Benchmarks with that dataset probably just need to rechunk right at the start.

[LDeakin]

Yeah this changed with zarr-developers/zarr-python#2463 for the zarr-python 3.0.0 release.

https://github.com/zarr-developers/zarr-python/blame/3b6565bd4377fc534a790a28e3436081e38c62b5/src/zarr/core/metadata/v3.py#L335-L340

[ilan-gold]

zarrs/zarrs-python#90 probably does not help too.

I was looking at that, I will look into it soon hopefully. Trying to prioritize for our next release at the moment (hence checking benchmarks etc.)

[ilan-gold]

I will run the full thing now, post the results, and then remove them if we're confident that these changes are good. Then you can run the benchmarks later on a machine whose specs you are perhaps more aware of (or we can keep what I post).

[LDeakin]

Thanks @ilan-gold!

zarrs / zarrs-python per seems to be within the noise, but some notable changes are:

• zarr-python has got a little quicker in the read all and roundtrip benchmarks
• tensorstore is a little quicker at reading the compressed dataset chunk-by-chunk
• The chunk-by-chunk dask benchmark is slower all around (dask changes?)

It'll be interesting to rerun benchmarks when zarr-python has both of these PRs released:

• Avoid memory copy in local store write zarr-developers/zarr-python#2944
• Improve write performance of shards zarr-developers/zarr-python#2977