fio results for RND4K Q32T16 are ~2x higher

[shvchk]

• Linux-distro (kernel version): Kubuntu 23.10, Linux 6.5.0-14-generic
• Desktop Environment (KDE/GNOME etc.): KDE
• Qt Version: 5.15.10
• KDiskMark Version: 3.1.3
• FIO Version: fio-3.35

Description:

Title says it all 😇 Other tests results like RND4K Q1T1 or SEQ1MQ8T1 are ~same as with fio. JSON files produced by fio (jobs.read.bw and jobs.write.bw) contain the same ~2x value, so maybe KDM calculation bug?

KDM

                        KDiskMark (3.1.3): https://github.com/JonMagon/KDiskMark
                    Flexible I/O Tester (fio-3.35): https://github.com/axboe/fio
--------------------------------------------------------------------------------
* MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]
* KB = 1000 bytes, KiB = 1024 bytes

[Read]
Sequential   1 MiB (Q=  8, T= 1):     0.000 MB/s [      0.0 IOPS] <     0.00 us>
Sequential 128 KiB (Q= 32, T= 1):     0.000 MB/s [      0.0 IOPS] <     0.00 us>
    Random   4 KiB (Q= 32, T=16):   477.734 MB/s [ 119434.4 IOPS] <  1071.35 us>
    Random   4 KiB (Q=  1, T= 1):     0.000 MB/s [      0.0 IOPS] <     0.00 us>

[Write]
Sequential   1 MiB (Q=  8, T= 1):     0.000 MB/s [      0.0 IOPS] <     0.00 us>
Sequential 128 KiB (Q= 32, T= 1):     0.000 MB/s [      0.0 IOPS] <     0.00 us>
    Random   4 KiB (Q= 32, T=16):   340.248 MB/s [  85062.9 IOPS] <  1503.83 us>
    Random   4 KiB (Q=  1, T= 1):     0.000 MB/s [      0.0 IOPS] <     0.00 us>

Profile: Default
   Test: 1 GiB (x5) [Measure: 5 sec / Interval: 5 sec]
   Date: 2024-01-03 01:10:59
     OS: ubuntu 23.10 [linux 6.5.0-14-generic]

fio

Read:

randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
...
fio-3.35
Starting 16 processes
Jobs: 16 (f=16): [r(16)][100.0%][r=907MiB/s][r=232k IOPS][eta 00m:00s]
randread: (groupid=0, jobs=16): err= 0: pid=3735: Wed Jan  3 01:08:50 2024
  read: IOPS=232k, BW=905MiB/s (949MB/s)(22.1GiB/25001msec)
    slat (usec): min=2, max=3277, avg=68.00, stdev=92.65
    clat (usec): min=142, max=6241, avg=2142.19, stdev=684.45
     lat (usec): min=178, max=6332, avg=2210.18, stdev=695.64
    clat percentiles (usec):
     |  1.00th=[  766],  5.00th=[ 1090], 10.00th=[ 1287], 20.00th=[ 1549],
     | 30.00th=[ 1745], 40.00th=[ 1926], 50.00th=[ 2114], 60.00th=[ 2278],
     | 70.00th=[ 2474], 80.00th=[ 2704], 90.00th=[ 3064], 95.00th=[ 3326],
     | 99.00th=[ 3884], 99.50th=[ 4113], 99.90th=[ 4555], 99.95th=[ 4752],
     | 99.99th=[ 5145]
   bw (  KiB/s): min=887256, max=987448, per=100.00%, avg=926638.69, stdev=1252.69, samples=784
   iops        : min=221814, max=246862, avg=231659.67, stdev=313.17, samples=784
  lat (usec)   : 250=0.01%, 500=0.04%, 750=0.87%, 1000=2.47%
  lat (msec)   : 2=40.95%, 4=54.94%, 10=0.72%
  cpu          : usr=1.80%, sys=8.96%, ctx=5788782, majf=42, minf=848
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued rwts: total=5790024,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: bw=905MiB/s (949MB/s), 905MiB/s-905MiB/s (949MB/s-949MB/s), io=22.1GiB (23.7GB), run=25001-25001msec

Disk stats (read/write):
  loop0: ios=5762003/0, merge=0/0, ticks=3131037/0, in_queue=3131037, util=99.60%
randwrite: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32

Write:

randwrite: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
...
fio-3.35
Starting 16 processes
Jobs: 16 (f=16): [w(16)][100.0%][w=675MiB/s][w=173k IOPS][eta 00m:00s]
randwrite: (groupid=0, jobs=16): err= 0: pid=3773: Wed Jan  3 01:09:19 2024
  write: IOPS=173k, BW=674MiB/s (707MB/s)(16.5GiB/25004msec); 0 zone resets
    slat (usec): min=2, max=1819, avg=91.02, stdev=88.44
    clat (usec): min=76, max=6129, avg=2872.92, stdev=542.36
     lat (usec): min=101, max=6269, avg=2963.94, stdev=551.21
    clat percentiles (usec):
     |  1.00th=[ 1696],  5.00th=[ 2024], 10.00th=[ 2212], 20.00th=[ 2409],
     | 30.00th=[ 2573], 40.00th=[ 2704], 50.00th=[ 2868], 60.00th=[ 2999],
     | 70.00th=[ 3130], 80.00th=[ 3326], 90.00th=[ 3589], 95.00th=[ 3785],
     | 99.00th=[ 4228], 99.50th=[ 4424], 99.90th=[ 4752], 99.95th=[ 4883],
     | 99.99th=[ 5211]
   bw (  KiB/s): min=668312, max=714360, per=100.00%, avg=690856.98, stdev=636.29, samples=784
   iops        : min=167078, max=178590, avg=172714.24, stdev=159.07, samples=784
  lat (usec)   : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.02%
  lat (msec)   : 2=4.46%, 4=92.99%, 10=2.50%
  cpu          : usr=2.07%, sys=7.06%, ctx=4325331, majf=0, minf=220
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,4316736,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: bw=674MiB/s (707MB/s), 674MiB/s-674MiB/s (707MB/s-707MB/s), io=16.5GiB (17.7GB), run=25004-25004msec

Disk stats (read/write):
  loop0: ios=0/4295648, merge=0/0, ticks=0/3119026, in_queue=3119044, util=99.61%

Steps To Reproduce:

Script below creates an in-memory XFS image /dev/shm/test.img, mounts it on /tmp/mnt/test-fs and tests with fio. When fio tests finish, script pauses, so you can do KDiskmark test on the same mount. After that, press Enter and script will unmount and delete image.

#! /usr/bin/env bash

img="/dev/shm/test.img"
dest="/tmp/mnt/test-fs"

_clean() {
  umount "$dest"
  rm -f "$img"
}

_pre_mount() {
  _clean
  mkdir -p "$dest"
  truncate -s 2G "$img"
}

_mount_xfs() {
  mkfs.xfs "$img"
  mount -o noatime "$img" "$dest"
}

_fio_test() {
  fio \
    --ioengine=libaio \
    --randrepeat=0 \
    --refill_buffers \
    --end_fsync=1 \
    --direct=1 \
    --filename="$dest/fio-test" \
    --name=$1 \
    --size=1024m \
    --runtime=25 \
    --group_reporting \
    --zero_buffers=0 \
    --rw=$1 \
    --bs=$2 \
    --iodepth=$3 \
    --numjobs=$4

  sleep 3
}

_test() {
  fio \
    --create_only=1 \
    --filename="$dest/fio-test" \
    --size=1024m \
    --zero_buffers=0 \
    --name=prepare

  _fio_test randread   4k   32  16
  _fio_test randwrite  4k   32  16

  rm -f "$dest/fio-test"
}

trap _clean EXIT

_pre_mount
_mount_xfs
_test

read -rsp "Press Enter when finished testing" < /dev/tty; echo

[JonMagon]

@shvchk It seems that 5 seconds by default is not really enough.
Could you please repeat the measurement with 10 seconds or over in settings window?

[Syhong330]

Hello, @JonMagon
I also found this same problem at Ubuntu 22.04
I don't think it's a problem with runtime. Because I checked normal performance (higher) of disk in system monitor about 1000MB/s. however only 500 MB/s in KDiskMark windows.
I think there seems to be a problem in the process of calculating for multi-job test results.