The performance of blobfuse2 seems not improved compare with blobfuse1

[jejass]

https://github.com/Azure/azure-storage-fuse/wiki/Blobfuse2-Performance-Improvements, from this wiki, it indicates that the blobfuse2's performance has improvement compare with blobfuse1. But in my testing, the performance looks similiar.

I use below 2 Azure virtual machines to do test.

Ubuntu 18.04, blobfuse 1.4.5
mount command: blobfuse /blobfuse --tmp-path=/mnt/blobfusetmp --config-file=/home/jasoncao/fuse_connection.cfg -o attr_timeout=120 -o entry_timeout=120 -o negative_timeout=240

Ubuntu 22.04, blobfuse2 2.2.0
mount configuration:https://github.com/Azure/azure-storage-fuse/blob/main/sampleFileCacheConfig.yaml

they are all with 1CPU, 3.5GiB RAM. they are in the same virtual network. Below are performance test result.

1. fio --name=seqread1k --bs=1k --rw=read --size=1G --ioengine=libaio --iodepth=1 --direct=0
1804 blobfuse1 output

2204 blobfuse2 output

below fio commands are same as the wiki.
2.fio --randrepeat=1 --ioengine=libaio --gtod_reduce=1 --name=test--bs=4k --iodepth=64 --rw=rw --rwmixread=75 --size=4G --filename=testfile4G
1804 blobfuse1 output

2204 blobfuse2 output

3.fio --randrepeat=1 --ioengine=libaio --gtod_reduce=1 --name=test--bs=4k --iodepth=64 --readwrite=randrw --rwmixread=75 --size=4G --filename=testfile4G
1804 blobfuse1 output

2204 blobfuse2 output

From the test result, it looks like the performance of blobfuse2 is similiar to blobfuse1. May I know why there is no significant difference between blobfuse1 and blobfuse2? even test with the command that in the wiki?

[vibhansa-msft]

It might be something with the test environment or the network, as we have a pipeline setup which periodically checks on the perf part and in our observations Blobfuse2 has always performed better. Keeping that aside, what do you want to achieve by comparing these two? If you are trying to figure out which one to use, then I recommend you to go for Blobfuse2 because Blobfuse1 is out of active development now and there will be no bug fix or feature request taken up for that code now.

[jejass]

thanks for your response. Now cx is concerning about the performance improvement from blobfuse1 to blobfuse2. We need this information to convince cx using blobfuse2. But from current testing result, they may not consider using blobfuse2 even Azure Storage.

I am trying to do more test with different environment to convince cx that blobfuse2 can meet their scenarios. may I know if it is possible to share more details like what kind of operating system and hardware the pipeline used to test the blobfuse2? cx also wants to understand more about how to adjust their architecture to use maximum performance of blobfuse2.

[vibhansa-msft]

Sure, let me throw a couple of scenarios here and you can see how Blobfuse2 can help you:

• Try listing a container which has over a million blobs. You will observe Blobfuse2 is much faster then Blobfuse1 in this case.
• Try reading a 10GB file sequentially. with v1 it will be too slow. With v2 if you use block-cache solution you will be able to read it without any local disk usage.
• if you wish to mount only a subdirectory and not entire container its possible only in v2.
• For perf part also our regular tests are showing good amount of improvement compared to v1 but not sure why in your case you are not able to get similar numbers. Can you turn off logging in v2 and then see if that gives you some perf boost.

[jejass]

@vibhansa-msft thanks for your information, may I confirm for the item 1 and 2, do you mean test reading via 'fio' command? or do a real read of a file? if it is doing a real read, how to determine the time cost precisely?

[vibhansa-msft]

You can either use fio to read sequentially (as random reads are expected to be slower in all file-systems) or have your own script or code that reads a file sequentially. After opening the file you can take a timestamp and once you are done reading you can take another one and the time lapse between these two data points will tell how much time it took to read the file.