scylladb/diskplorer

Diskplorer - disk latency/bandwidth grapher

Diskplorer is a small set of tools around fio that can be used to discover disk read latency at different read and write workloads. Diskplorer runs a matrix of 21 write workloads (0% to 100% of the maximum bandwidth, in 5% increments) and 21 read workloads (0% to 100% of the maximum IOPS, in 5% increments) for a total of 441 different workloads. The disk is fully written first in order to eliminate clean-disk effects.

Diskplorer runs in two steps: step 1 generates a json result file, and must be run on the system being tested, and step 2 generates latency charts from the result file.

Running the test

Caution: the test is destructive. Do not use on disks that have real data.

Install the dependencies with

sudo dnf install -y fio

or

apt-get install -y fio

Obtain the maximum write bandwith and maximum read IOPS from the device data sheet.

Run diskplorer (substitute /dev/name with your device file):

sudo ./diskplorer.py /dev/name --result-file your-results.json

It is recommended to save the fio test file for later reference (--fio-job-directory)

sudo is required due to direct disk access.

This will run for several hours. Some smoke may be emitted from the disk.

Viewing the results

Once done, copy the result file (test.json) to your workstation and view the charts with

./latency-postprocess.py test.json

Sample results

i3en.3xlarge

With 4k blocksize, showing reduction in IOPS due to increased bandwidth demand:

With 16k blocksize, showing further reduction in IOPS due to increased bandwidth demand:

i3.2xlarge

im4gn.4xlarge

AWS EC2 r5b.2xlarge, EBS GP3 (1000 GB, 1000 MB/s, 16000 IOPS)

GCP n2-standard-16, 8 local SSDs in RAID 0

These results are using aio instead of io_uring due to bad interaction between md and io_uring.

GCP n2-standard-8, 2 TB SSD Persistent Disk

Strangely the 95th percentile at low rates is worse than at high rates.

Toshiba DT01ACA200 hard disk drive

Results for a rotating hard disk drive. Note the throughput and IOPS were allows to miss by a 15% margin rather than the normal 3% margin.