Idea: Per-file system tiering statistics (e.g. download/upload at intervals)

[jmcvaughn]

From @adamuk01, who will be able to correct me for this first Weka GitHub issue!

It is possible to obtain per-object-store metrics (e.g. amount of data uploaded/downloaded) for tiering. However, it is not possible to obtain this data per file system.

This could be generated by collating per-object upload/download metrics, and would give a better idea of total utilisation of each file system and NVMe tier hits/misses for further policy tuning as appropriate.

[vince-weka]

Can Adam provide commands used to get these stats?

[jmcvaughn]

Ah, thought I mentioned: weka fs tier ops <name> is the main one. Example output:

weka fs tier ops example-bucket
NODE      OBJECT STORE IDENTIFIER                                        OPERATION TYPE     EXECUTION TIME     PHASE                   LAST EXECUTION TIME     START TIME               SIZE         PREVIOUS ATTEMPT RESULTS     CURL ERRORS     CONCURRENCY LEVEL     INODE ID
2         B_9612/2635b321-65a1-4175-bfd0-94a5c7b938e9/s/I_0/228425/0     Upload             0.21s              Recent                  0.21s                   2022-Oct-06 02:00:41     56           SUCCESS: 1                                   5                     INVALID
2         B_6163/2635b321-65a1-4175-bfd0-94a5c7b938e9/s/I_0/228334/0     Upload             0.22s              Recent                  0.22s                   2022-Oct-06 02:00:41     2740         SUCCESS: 1                                   6                     INVALID

In its most basic form, add up all the SIZE field's values for a particular time period, but due to my lack of familiarity at the moment I'm not sure if OBJECT STORE IDENTIFIER contains any references to a file system (given the /s in the path perhaps it might); something to learn more about tomorrow!