[wsl2] filesystem performance is much slower than wsl1 in /mnt #4197

ioweb-gr · 2019-06-19T22:29:56Z

I decided to open this as a separate issue because although it's related to the generic issue of filesystem performance it's directly related to WSL 2 while the other issues are for WSL 1 and it's showing very conflicting results.

Your Windows build number: (Type ver at a Windows Command Prompt)
Version 10.0.18917 Build 18917
What you're doing and what's happening:
I'm testing filesystem write speed in /mnt using dd command. Performing the following tests

WSL2

root@LUCIANO-PC:/home/# dd if=/dev/zero of=/mnt/e/testfile bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 25.939 s, 40.4 MB/s

WSL1

root@LUCIANO-PC:/home/# dd if=/dev/zero of=/mnt/e/testfile bs=1M count=20000
20000+0 records in
20000+0 records out
20971520000 bytes (21 GB, 20 GiB) copied, 47.4897 s, 442 MB/s

On / it's actually the reverse. WSL2 is more than 2 times faster than WSL1.

What's wrong / what should be happening instead:
I would expect the filesystem performance in /mnt to at least be on the same level but it's over 10 times slower.

Another interesting fact is that if I mount the same drive as a cifs share I get 3x performance

WSL 2 (cifs share)

root@LUCIANO-PC:/mnt/sambae# dd if=/dev/zero of=/mnt/sambae/testfile bs=1M count=10000
10000+0 records in
10000+0 records out
10485760000 bytes (10 GB, 9.8 GiB) copied, 84.001 s, 125 MB/s

An update of the current status because it's way too hidden in this thread.

Latest status report: #4197 (comment)_

The text was updated successfully, but these errors were encountered:

billziss-gh · 2019-06-20T04:01:34Z

On / it's actually the reverse. WSL2 is more than 2 times faster than WSL1.

Can you post the results when doing the same test on a file on / for WSL1 and WSL2?

This is straight I/O on a single file where I would expect to see more or less the same perf between WSL1 and WSL2. My expectation was that WSL2 perf is better than WSL1 only when doing "namespace" operations (i.e. working on lots of small files, listing them, stat'ing them, etc.)

As for the results on /mnt/e they are not very surprising, since I/O has to go through both the Linux and Windows file system stack in likely a new and unoptimized piece of code.

ioweb-gr · 2019-06-20T09:35:06Z

I see,

I'm assuming you're still refering to / and not /mnt folders about namespace operations.
I've repeated the write tests in both environments a few times now and there's no consistent result.

WSL 2

root@LUCIANO-PC:/home/# dd if=/dev/zero of=/testfile bs=1M count=10000
10000+0 records in
10000+0 records out
10485760000 bytes (10 GB, 9.8 GiB) copied, 7.0873 s, 1.5 GB/s
root@LUCIANO-PC:/home/# dd if=/dev/zero of=/testfile bs=1M count=20000

20000+0 records in
20000+0 records out
20971520000 bytes (21 GB, 20 GiB) copied, 51.3779 s, 408 MB/s
root@LUCIANO-PC:/home/#
root@LUCIANO-PC:/home/# dd if=/dev/zero of=~/testfile bs=1M count=20000
^C12360+0 records in
12360+0 records out
12960399360 bytes (13 GB, 12 GiB) copied, 12.1766 s, 1.1 GB/s

WSL1

root@LUCIANO-PC:/home/# dd if=/dev/zero of=~/testfile bs=1M count=10000
10000+0 records in
10000+0 records out
10485760000 bytes (10 GB, 9.8 GiB) copied, 29.7096 s, 353 MB/s
root@LUCIANO-PC:/home/#

I'm thinking this has to do something with OS Caching because in task manager even though I can see the file being written, the disk usage write speed is unchanged while in wsl1 I can see it constantly at around 350mb/sec .

As another interesting fact, the vmmem usage while writing the file is increasing rapidly. By the 3rd copy action all my RAM is used up (32GB).
It goes back down a while after quitting my shell. So maybe the performance is bottlenecked by this bug as well.

paultyng · 2019-07-12T19:08:21Z

I'm also experiencing this after an upgrade to WSL2, only slow performance under /mnt, not /, slower than WSL1. I can post numbers as well if necessary. This is on Microsoft Windows [Version 10.0.18936.1000].

definelicht · 2019-07-16T00:24:32Z

I'm experiencing the same issue, /mnt is extremely slow compared to WSL1 (seconds to run git status on a relatively small repository).

paultyng · 2019-07-18T15:29:34Z

When I run a git status on a clone of https://github.com/hashicorp/terraform in /mnt I get output like this:

It took 6.97 seconds to enumerate untracked files. 'status -uno'
may speed it up, but you have to be careful not to forget to add
new files yourself (see 'git help status').

ioweb-gr · 2019-07-19T06:51:17Z

This is an example from the command

pv files.tar.gz | tar -zxf -

The speed is amazingly slow in wsl2 on /mnt

definelicht · 2019-07-22T16:38:30Z

Do we know what's at the root of this, and whether there are any other workarounds than downgrading?

benhillis · 2019-07-22T16:52:51Z

@definelicht - Yep we're working on improving the performance. In the meaning working out of your root file system (the ext4 volume) will have MUCH better performance.

ioweb-gr · 2019-07-27T06:19:42Z

Indeed after the last update I can already see much better performance on WSL2.

root@LUCIANO-PC:~# dd if=/dev/zero of=/mnt/e/testfile bs=1M count=10000
10000+0 records in
10000+0 records out
10485760000 bytes (10 GB, 9.8 GiB) copied, 41.277 s, 254 MB/s

That's on an SSD and it's like 6 times faster than before. Still slower then WSL1 but it's definitely improved.

--correction---
I see much better performance on sequential writes. Untar seems very slow still extracting data at bytes/sec while in wsl1 I can see speeds at kb/sec for the same part

tuananh · 2019-08-12T04:16:04Z

Indeed after the last update I can already see much better performance on WSL2.

root@LUCIANO-PC:~# dd if=/dev/zero of=/mnt/e/testfile bs=1M count=10000
10000+0 records in
10000+0 records out
10485760000 bytes (10 GB, 9.8 GiB) copied, 41.277 s, 254 MB/s

That's on an SSD and it's like 6 times faster than before. Still slower then WSL1 but it's definitely improved.

--correction---
I see much better performance on sequential writes. Untar seems very slow still extracting data at bytes/sec while in wsl1 I can see speeds at kb/sec for the same part

I got


1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 3.57894 s, 293 MB/s

but git status still taking more than 2 seconds on quite small repo

definelicht · 2019-08-12T04:18:20Z

I think this is a latency rather than a bandwidth issue, surfacing when a large number of files are accessed (e.g., when running git status).

tuananh · 2019-08-14T03:07:39Z

@definelicht - Yep we're working on improving the performance. In the meaning working out of your root file system (the ext4 volume) will have MUCH better performance.

can you share the cause of it? I'm curious

gbatalski · 2019-08-15T17:30:12Z

can confirm a very bad performance on /mnt (mvn install takes forever), on /tmp it takes only 2 minutes for the same project

nono-pxbsd · 2019-09-11T05:53:18Z

After switch configuration from WSL to WSL 2 (Build 18975), without any change on my distro, the server respond time incresed.

Configuration : Debian buster Apache2.X / PHP-FPM 7.3 / MySQL
Root : /
Web directories : /c/repositories/xxx

WSL 1 : 0.5 sec
WSL 2 : 8 sec

With Docker (Ubuntu 18.04) WSL Tech preview i could not test on WSL 1, but WSL 2 takes over 30 sec... I suppose the issue is the same.

realtica · 2019-09-13T18:42:48Z

This is my performance in a medium project:

GIT_TRACE_PERFORMANCE=1 git status -sb -uno

13:20:04.630972 read-cache.c:2267 13:20:07.766270 preload-index.c:146 Refresh index: 100% (3463/3463), 13:20:53.531359 read-cache.c:1575 13:20:53.981829 read-cache.c:2267 13:20:54.383135 read-cache.c:1575 13:20:54.426399 diff-lib.c:251 13:20:54.468000 unpack-trees.c:1554 13:20:54.468266 unpack-trees.c:465 13:20:54.468346 unpack-trees.c:1649 13:20:54.468368 diff-lib.c:537 13:20:54.569793 read-cache.c:2989 13:20:54.586550 trace.c:477 13:20:55.277184 read-cache.c:2267 13:20:55.958706 read-cache.c:1575 13:20:56.007094 diff-lib.c:251 13:20:56.041768 unpack-trees.c:1554 13:20:56.042014 unpack-trees.c:465 13:20:56.042084 unpack-trees.c:1649 13:20:56.215068 trace.c:477 performance: 0.017249300 s: read cache .git/index
performance: 3.135180100 s: preload index
done.
performance: 48.900269600 s: refresh index
performance: 0.016242500 s: read cache /mnt/e/p/keyed_au/.git/ modules/Files/media/nusoap/index
performance: 0.401222700 s: refresh index
performance: 0.000005000 s: diff-files
performance: 0.000087700 s: traverse_trees
performance: 0.000164900 s: check_updates
performance: 0.000438000 s: unpack_trees
performance: 0.001873400 s: diff-index
performance: 0.014679900 s: write index, changed mask = 2
performance: 0.736638600 s: git command: /usr/lib/git-core/git status --porcelai n=2 -uno
performance: 0.031572000 s: read cache /mnt/e/p/keyed_au/.git/ modules/Files/modules/unittest/index
performance: 0.681302500 s: refresh index
performance: 0.000003500 s: diff-files
performance: 0.000081200 s: traverse_trees
performance: 0.000162100 s: check_updates
performance: 0.000399800 s: unpack_trees 13:20:56.042102 diff-lib.c:537 performance: 0.000454400 s: diff-index 13:20:56.126615 read-cache.c:2989 performance: 0.012095500 s: write index, changed mask = 2 13:20:56.141174 trace.c:477 performance: 1.261971900 s: git command: /usr/lib/git-core/git status --porcelain=2 -uno 13:20:56.148677 diff-lib.c:251 performance: 2.586112000 s: diff-files 13:20:56.190888 unpack-trees.c:1554 performance: 0.000022200 s: traverse_trees 13:20:56.191028 unpack-trees.c:465 performance: 0.000027800 s: check_updates 13:20:56.191106 unpack-trees.c:1649 performance: 0.000639200 s: unpack_trees 13:20:56.191175 diff-lib.c:537 performance: 0.001413100 s: diff-index
performance: 51.628126000 s: git command: git status -sb -uno

File System exFat, the result is the same is other file system...
WSL 2 git=51.62 sec
git for windows=0.8 sec

paslandau · 2019-09-15T13:56:45Z

I'm currently using the normal Docker Desktop with linux containers to run a PHP projekt during local development. The repo is in a normal Windows directory and gets mounted via volume bind in the docker container, so that I can easily manipulate the files through an IDE.

The same setup on a native linux machine has always been multiple times faster and the "problem" (from what I've read) has always been attributed to the Linux VM that is used on Docker Desktop.

Then WSL 2 was announced and I finally came around to set everything up. But when I ran our teststuite for the first time, I was pretty disappointed that the performance was actually much worse than before (40s on the Docker Deskop Linux VM setup vs 100s on the WSL2 setup). Subsequent investigation brought me to this thread.

From what I've read here so far:

Does it even make sense to switch to WSL 2 for local development yet if "bad performance" is your primary reason to do so?

A "no" would be a totally acceptable answer - I just don't want to waste any more time on this if it's simply not ready yet.

stweedie · 2019-09-19T18:18:03Z

I know that wsl2 is a preview, but is there any hope that accessing \wsl$ from windows or /mnt from wsl will reach acceptable speeds?

gormulent · 2019-09-24T01:58:51Z

I'd also like to ask about this, doing a git status on something in /mnt/c/.. takes about a minute, something seems wrong ..

gbatalski · 2019-09-27T06:32:53Z

same with 1903 (18990.1) absolutely useless...

wusticality · 2019-09-27T11:05:32Z

I am also seeing this, accessing Windows files from the Linux environment is several orders of magnitude slower in WSL2 than in WSL1.

ericblade · 2024-02-23T05:52:25Z

Would it be safe to say that using CIFS instead of DRVFS for the windows mounts, would be a better idea, performance wise?

elismasilva · 2024-02-23T13:10:54Z

Is there any way to perform a Linux partition mount on Windows? or accessed it in some way? What I thought might be a solution: I dedicated an nvme to create a linux partition directly through ubuntu and so it would have access through the linux kernel and not through the mount point. The problem would be after I access these files from windows in a fast way like NTFS. Or perhaps it is not possible to mount directly in NTFS without using the Windows mount. What do you think would work?

elismasilva · 2024-02-23T13:40:57Z

CIFS

I tested this now and I didn't see any improvement, the same speed, I believe the problem is the fact that it travels across the network regardless of the form of sharing.

muness · 2024-02-23T13:45:25Z

Is there any way to perform a Linux partition mount on Windows? or accessed it in some way? What I thought might be a solution: I dedicated an nvme to create a linux partition directly through ubuntu and so it would have access through the linux kernel and not through the mount point. The problem would be after I access these files from windows in a fast way like NTFS. Or perhaps it is not possible to mount directly in NTFS without using the Windows mount. What do you think would work?

I've had success with this (commercial) driver to mount extfs partitions on windows : https://www.paragon-software.com/home/linuxfs-windows/

I mount the partition in the wsl instance as described at https://www.bleepingcomputer.com/news/microsoft/windows-10-now-lets-you-mount-linux-ext4-filesystems-in-wsl-2/

muness · 2024-02-23T15:10:03Z

Alas, the paragon extfs driver isn't useful in addressing the performance issue:

the performance of the Paragon extfs driver is 10x as bad...
changes from Windows don't show up in WSL and vice versa until the partition is remounted.

Cloning the git repo on the extfs drive from windows and then checking status

PS E:\muness> gh repo clone git/git
Cloning into 'git'...
remote: Enumerating objects: 361613, done.
remote: Counting objects: 100% (156/156), done.
remote: Compressing objects: 100% (67/67), done.
remote: Total 361613 (delta 98), reused 142 (delta 89), pack-reused 361457
Receiving objects: 100% (361613/361613), 232.37 MiB | 5.03 MiB/s, done.
Resolving deltas: 100% (272082/272082), done.
Updating files: 100% (4433/4433), done.

PS E:\muness\git> $env:GIT_TRACE_PERFORMANCE = 1
PS E:\muness\git> git status -sb -uno | grep performance
...
A  trace2/10:05:47.235100 trace.c:414             performance: 1.679887300 s: git command: git.exe status -sb -uno

vs cloning it and checking status from WSL:

➜  ~ cd /mnt/wsl/PHYSICALDRIVE2p2/
(.venv) ➜  muness gh repo clone git/git
Cloning into 'git'...
remote: Enumerating objects: 361613, done.
remote: Counting objects: 100% (156/156), done.
remote: Compressing objects: 100% (67/67), done.
remote: Total 361613 (delta 98), reused 142 (delta 89), pack-reused 361457
Receiving objects: 100% (361613/361613), 232.08 MiB | 55.25 MiB/s, done.
Resolving deltas: 100% (272061/272061), done.
(.venv) ➜  muness cd git
(.venv) ➜  git git:(master) GIT_TRACE_PERFORMANCE=1 git status -sb -uno
10:00:38.331662 read-cache.c:2398       performance: 0.000471924 s:  read cache .git/index
10:00:38.334786 preload-index.c:154     performance: 0.002999047 s:   preload index
10:00:38.450301 read-cache.c:1690       performance: 0.118514726 s:  refresh index
10:00:38.450516 diff-lib.c:266          performance: 0.000078137 s:  diff-files
10:00:38.450845 unpack-trees.c:1802     performance: 0.000043702 s:    traverse_trees
10:00:38.450875 unpack-trees.c:418      performance: 0.000000251 s:    check_updates
10:00:38.450879 unpack-trees.c:1892     performance: 0.000101310 s:   unpack_trees
10:00:38.450893 diff-lib.c:629          performance: 0.000287459 s:  diff-index
10:00:38.452116 read-cache.c:3072       performance: 0.001206149 s:  write index, changed mask = 0
## master...origin/master
10:00:38.452412 trace.c:487             performance: 0.121396593 s: git command: git status -sb -uno

elismasilva · 2024-02-23T17:10:35Z

Alas, the paragon extfs driver isn't useful in addressing the performance issue:

the performance of the Paragon extfs driver is 10x as bad...
changes from Windows don't show up in WSL and vice versa until the partition is remounted.

Cloning the git repo on the extfs drive from windows and then checking status

PS E:\muness> gh repo clone git/git
Cloning into 'git'...
remote: Enumerating objects: 361613, done.
remote: Counting objects: 100% (156/156), done.
remote: Compressing objects: 100% (67/67), done.
remote: Total 361613 (delta 98), reused 142 (delta 89), pack-reused 361457
Receiving objects: 100% (361613/361613), 232.37 MiB | 5.03 MiB/s, done.
Resolving deltas: 100% (272082/272082), done.
Updating files: 100% (4433/4433), done.

PS E:\muness\git> $env:GIT_TRACE_PERFORMANCE = 1
PS E:\muness\git> git status -sb -uno | grep performance
...
A  trace2/10:05:47.235100 trace.c:414             performance: 1.679887300 s: git command: git.exe status -sb -uno

vs cloning it and checking status from WSL:

➜  ~ cd /mnt/wsl/PHYSICALDRIVE2p2/
(.venv) ➜  muness gh repo clone git/git
Cloning into 'git'...
remote: Enumerating objects: 361613, done.
remote: Counting objects: 100% (156/156), done.
remote: Compressing objects: 100% (67/67), done.
remote: Total 361613 (delta 98), reused 142 (delta 89), pack-reused 361457
Receiving objects: 100% (361613/361613), 232.08 MiB | 55.25 MiB/s, done.
Resolving deltas: 100% (272061/272061), done.
(.venv) ➜  muness cd git
(.venv) ➜  git git:(master) GIT_TRACE_PERFORMANCE=1 git status -sb -uno
10:00:38.331662 read-cache.c:2398       performance: 0.000471924 s:  read cache .git/index
10:00:38.334786 preload-index.c:154     performance: 0.002999047 s:   preload index
10:00:38.450301 read-cache.c:1690       performance: 0.118514726 s:  refresh index
10:00:38.450516 diff-lib.c:266          performance: 0.000078137 s:  diff-files
10:00:38.450845 unpack-trees.c:1802     performance: 0.000043702 s:    traverse_trees
10:00:38.450875 unpack-trees.c:418      performance: 0.000000251 s:    check_updates
10:00:38.450879 unpack-trees.c:1892     performance: 0.000101310 s:   unpack_trees
10:00:38.450893 diff-lib.c:629          performance: 0.000287459 s:  diff-index
10:00:38.452116 read-cache.c:3072       performance: 0.001206149 s:  write index, changed mask = 0
## master...origin/master
10:00:38.452412 trace.c:487             performance: 0.121396593 s: git command: git status -sb -uno

But what is the performance in reading and writing large files like 1GB or more in Python for example or even copying large files in folders on ext4 partitions in WSL2? my problem today is accessing these large files on mount points for windows drives. If ext4 on Linux is as fast as the virtual drive on the WSL2 system, then the last step is to figure out how to access that Linux partition directly from Windows and copy those same files without losing performance.

muness · 2024-02-23T17:24:32Z

But what is the performance in reading and writing large files like 1GB or more in Python for example or even copying large files in folders on ext4 partitions in WSL2? my problem today is accessing these large files on mount points for windows drives. If ext4 on Linux is as fast as the virtual drive on the WSL2 system, then the last step is to figure out how to access that Linux partition directly from Windows and copy those same files without losing performance.

My impression of the /mnt/wsl/PHYSICALDRIVE2p2/ mount point is that it's as fast as the using the WSL image. I shared a git example above of the external WSL mount at 232.08 MiB | 55.25 MiB/ for git. Here are some examples with dd.

Using Paragon Linux Filesystem:

root@um790pro:~# dd if=/dev/zero of=/mnt/f/testfile bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 17.1656 s, 61.1 MB/s

Or using the WSL extfs mount:

root@um790pro:~# dd if=/dev/zero of=/mnt/wsl/PHYSICALDRIVE2p3/testfile bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 0.476176 s, 2.2 GB/s

On the image:

root@um790pro:~# dd if=/dev/zero of=./testfile bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 0.481858 s, 2.2 GB/s

On the WSL mount of the NTFS drive:

root@um790pro:~# dd if=/dev/zero of=/mnt/c/Users/munes/testfile bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 5.17415 s, 203 MB/s

I'd be happy to run other tests.

muness · 2024-02-23T17:35:52Z

then the last step is to figure out how to access that Linux partition directly from Windows and copy those same files without losing performance.

This has me stumped: I don't know of a fast extfs driver for Windows.

fredizzimo · 2024-02-23T18:47:18Z

then the last step is to figure out how to access that Linux partition directly from Windows and copy those same files without losing performance.

This has me stumped: I don't know of a fast extfs driver for Windows.

I don't think that will work; the file system is not meant to be used by two drivers/kernels at the same time. So even if you get it to work, it will most likely corrupt the file system. Or if one of them is just read only, it will still most likely just read stale data, due to the file caches not being in sync.

And that's the whole reason why they use a network file system in the first place, otherwise WSL would already use direct mounts on both sides.

But...
We are still waiting to see the impact of the optimizations that were added to the 6.1 kernel in the end of 2022 https://www.phoronix.com/news/Linux-9p-10x-Performance, but WSL is still using kernel 5.15 for some reason. I haven't seen any benchmarks with a custom-built kernel, but I don't think it's enough, we probably need a similar patch on the Windows side as well.

fredizzimo · 2024-02-23T20:13:30Z

I did some more digging.

There has actually been quite a bit of progress of writing custom 9P drivers for QEMU https://gitlab.com/qemu-project/qemu/-/issues/974, with links to various mailing list discussions. If that was completed, I think it should be usable together with WSL2 as well.

Unfortunately, it looks like the progress has stalled due to security issues.

For virtiofsd, which probably would be faster, the situation doesn't look as promising https://gitlab.com/qemu-project/qemu/-/issues/623. Windows guest support exist, but I don't think the drivers can be installed and work on a physical machine. So, we need host driver support, where WSL would be a guest.

elismasilva · 2024-02-24T03:28:37Z

Encontrei uma solução alternativa para anexar um volume formatado para linux: https://anthony-f-tannous.medium.com/wsl2-how-to-prepare-and-attach-virtual-drives-vhd-ac17b1fc7a61

Mas tentar usá-lo para /home ou /home/odie5533 não funcionou para mim. VS Code/WSL2 lançaria um ajuste ao tentar começar, já que não seria montado corretamente. Ele começou a fazer arquivos fantasmas na partição / em /home/odie5533/vscodestuff, mesmo que eu quisesse ter o volume montado lá. Foi muito trabalho para configurar, então é uma que não funcionou.

Trago isso porque poderia ser útil para as pessoas em algumas circunstâncias, particularmente para a montagem de volumes externos. Provavelmente não sofreria com os problemas de velocidade que as pessoas estão vendo aqui.

Is there any way to perform a Linux partition mount on Windows? or accessed it in some way? What I thought might be a solution: I dedicated an nvme to create a linux partition directly through ubuntu and so it would have access through the linux kernel and not through the mount point. The problem would be after I access these files from windows in a fast way like NTFS. Or perhaps it is not possible to mount directly in NTFS without using the Windows mount. What do you think would work?

I've had success with this (commercial) driver to mount extfs partitions on windows : https://www.paragon-software.com/home/linuxfs-windows/

I mount the partition in the wsl instance as described at https://www.bleepingcomputer.com/news/microsoft/windows-10-now-lets-you-mount-linux-ext4-filesystems-in-wsl-2/

I finally got this working with an ext4 vhdx and it actually works really fast. The only problem is that to access via Windows you need to access the Linux mount point as is done when viewing the contents of the WSL distribution disk. So I can only access the path '\wsl.localhost\distro\home\user' if Linux is running. I'll try to repeat creating a vhdx in ntfs format, because if it works I'll be able to mount the vhdx in Windows when I'm not using it within WSL. Then I will structure the step by step and send the link here for anyone who wants to try it.

elismasilva · 2024-02-24T16:07:08Z

But what is the performance in reading and writing large files like 1GB or more in Python for example or even copying large files in folders on ext4 partitions in WSL2? my problem today is accessing these large files on mount points for windows drives. If ext4 on Linux is as fast as the virtual drive on the WSL2 system, then the last step is to figure out how to access that Linux partition directly from Windows and copy those same files without losing performance.

My impression of the /mnt/wsl/PHYSICALDRIVE2p2/ mount point is that it's as fast as the using the WSL image. I shared a git example above of the external WSL mount at 232.08 MiB | 55.25 MiB/ for git. Here are some examples with dd.

Using Paragon Linux Filesystem:
root@um790pro:~# dd if=/dev/zero of=/mnt/f/testfile bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 17.1656 s, 61.1 MB/s
Or using the WSL extfs mount:
root@um790pro:~# dd if=/dev/zero of=/mnt/wsl/PHYSICALDRIVE2p3/testfile bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 0.476176 s, 2.2 GB/s
On the image:
root@um790pro:~# dd if=/dev/zero of=./testfile bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 0.481858 s, 2.2 GB/s
On the WSL mount of the NTFS drive:
root@um790pro:~# dd if=/dev/zero of=/mnt/c/Users/munes/testfile bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 5.17415 s, 203 MB/s
I'd be happy to run other tests.

Ive tested NTFS mount and it is really only 110mb/s transfer speed while in ext4 i got 2gb/s like direct nvme writing.

See: microsoft/WSL#4197

leonpano2006 · 2024-04-15T07:45:23Z

when i tranfser file to xfs via wsl2 and i fell like my speed is often limited around 1Gbps(100 to 125MiB/s) or 2Gbps(180 to 200MiB/s)

pocesar · 2024-04-24T19:10:50Z

Docker build on /root is 4.2s (EVO 970 ~3500MB/s), on /mnt is 608s (/mnt is mounted on a NVMe 7000MB/s)

See: microsoft/WSL#4197

foriequal0 · 2024-06-26T16:45:03Z

But... We are still waiting to see the impact of the optimizations that were added to the 6.1 kernel in the end of 2022 phoronix.com/news/Linux-9p-10x-Performance, but WSL is still using kernel 5.15 for some reason. I haven't seen any benchmarks with a custom-built kernel, but I don't think it's enough, we probably need a similar patch on the Windows side as well.

6.1 kernel doesn't make a huge change.
NTFS mount on WSL2

5.15.153.1-microsoft-standard-WSL2
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 3.76512 s, 278 MB/s

6.1.21.2-microsoft-standard-WSL2+
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 3.73518 s, 281 MB/s

Compare to ext4 on WSL2

5.15.153.1-microsoft-standard-WSL2
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 0.361747 s, 2.9 GB/s

See: microsoft/WSL#4197

therealkenc added the discussion label Jun 20, 2019

therealkenc mentioned this issue Jun 23, 2019

Accessing Linux files through \\wsl$ is unbearably slow #4220

Closed

Biswa96 mentioned this issue Jul 29, 2019

Write Files to Windows Host is Too Slow #4352

Closed

This was referenced Aug 8, 2019

Running docker on Ubuntu 18.04.2 LTS (WSL Enabled) #4385

Closed

Docker containers are too slow on WSL 2 #4387

Closed

therealkenc mentioned this issue Aug 12, 2019

git status is slow in WSL2 #4401

Closed

tuananh mentioned this issue Aug 22, 2019

Latency justjanne/powerline-go#129

Open

therealkenc mentioned this issue Sep 7, 2019

Filesystem Feature and Performance Issues #4478

Closed

craigloewen-msft mentioned this issue Sep 17, 2019

WSL2: File operations on NTFS folders extremely slow #4515

Closed

Biswa96 mentioned this issue Sep 24, 2019

Access windows git project with wsl git is too slow #4540

Closed

jenbroek added a commit to jenbroek/etc that referenced this issue Mar 6, 2024

invoke git.exe instead of git in /mnt

78805a0

See: microsoft/WSL#4197

jenbroek added a commit to jenbroek/etc that referenced this issue Mar 9, 2024

invoke git.exe instead of git in /mnt

803c318

See: microsoft/WSL#4197

jenbroek added a commit to jenbroek/etc that referenced this issue Mar 9, 2024

invoke git.exe instead of git in /mnt

d660ca0

See: microsoft/WSL#4197

aliencaocao mentioned this issue Mar 23, 2024

Extremely slow model loading for BNB 4bit saved weights huggingface/transformers#29810

Closed

4 tasks

jenbroek added a commit to jenbroek/etc that referenced this issue Mar 26, 2024

invoke git.exe instead of git in /mnt

21709a1

See: microsoft/WSL#4197

edumserrano mentioned this issue Apr 8, 2024

Please reconsider adding support for --poll option for ng build when building libraries angular/angular-cli#27420

Closed

lukasf mentioned this issue Apr 25, 2024

Add support for the new NTFS3 kernel module #8564

Open

Avasam mentioned this issue May 10, 2024

Speedup pre-commit run by excluding python files from trailing hooks python/typeshed#11777

Closed

jenbroek added a commit to jenbroek/etc that referenced this issue Jun 4, 2024

invoke git.exe instead of git in /mnt

a98de4b

See: microsoft/WSL#4197

sceneq mentioned this issue Jun 19, 2024

rg.exe does not complete in bash.exe(WSL2) - Incorrectly assumes stdin BurntSushi/ripgrep#2838

Closed

1 task

Avasam mentioned this issue Aug 11, 2024

Rewriting proto scripts in Python python/typeshed#12511

Closed

jenbroek added a commit to jenbroek/etc that referenced this issue Sep 11, 2024

invoke git.exe instead of git in /mnt

4caecc2

See: microsoft/WSL#4197

jenbroek added a commit to jenbroek/etc that referenced this issue Sep 24, 2024

invoke git.exe instead of git in /mnt

7b61d15

See: microsoft/WSL#4197

This was referenced Sep 25, 2024

Windows support triton-lang/triton#4045

Closed

Investigate torch.compile Windows support. pytorch/pytorch#122094

Open

[wsl2] filesystem performance is much slower than wsl1 in /mnt #4197

[wsl2] filesystem performance is much slower than wsl1 in /mnt #4197

Comments

ioweb-gr commented Jun 19, 2019 • edited Loading

WSL2

WSL1

WSL 2 (cifs share)

An update of the current status because it's way too hidden in this thread.

billziss-gh commented Jun 20, 2019

ioweb-gr commented Jun 20, 2019 • edited Loading

WSL 2

WSL1

paultyng commented Jul 12, 2019

definelicht commented Jul 16, 2019

paultyng commented Jul 18, 2019

ioweb-gr commented Jul 19, 2019 • edited Loading

definelicht commented Jul 22, 2019

benhillis commented Jul 22, 2019 • edited Loading

ioweb-gr commented Jul 27, 2019 • edited Loading

tuananh commented Aug 12, 2019

definelicht commented Aug 12, 2019

tuananh commented Aug 14, 2019

gbatalski commented Aug 15, 2019 • edited Loading

nono-pxbsd commented Sep 11, 2019 • edited Loading

realtica commented Sep 13, 2019

paslandau commented Sep 15, 2019 • edited Loading

stweedie commented Sep 19, 2019

gormulent commented Sep 24, 2019

gbatalski commented Sep 27, 2019

wusticality commented Sep 27, 2019

ericblade commented Feb 23, 2024

elismasilva commented Feb 23, 2024

elismasilva commented Feb 23, 2024

muness commented Feb 23, 2024

muness commented Feb 23, 2024 • edited Loading

elismasilva commented Feb 23, 2024

muness commented Feb 23, 2024 • edited Loading

muness commented Feb 23, 2024

fredizzimo commented Feb 23, 2024

fredizzimo commented Feb 23, 2024

elismasilva commented Feb 24, 2024

elismasilva commented Feb 24, 2024

leonpano2006 commented Apr 15, 2024 • edited Loading

pocesar commented Apr 24, 2024

foriequal0 commented Jun 26, 2024 • edited Loading

ioweb-gr commented Jun 19, 2019 •

edited

Loading

ioweb-gr commented Jun 20, 2019 •

edited

Loading

ioweb-gr commented Jul 19, 2019 •

edited

Loading

benhillis commented Jul 22, 2019 •

edited

Loading

ioweb-gr commented Jul 27, 2019 •

edited

Loading

gbatalski commented Aug 15, 2019 •

edited

Loading

nono-pxbsd commented Sep 11, 2019 •

edited

Loading

paslandau commented Sep 15, 2019 •

edited

Loading

muness commented Feb 23, 2024 •

edited

Loading

muness commented Feb 23, 2024 •

edited

Loading

leonpano2006 commented Apr 15, 2024 •

edited

Loading

foriequal0 commented Jun 26, 2024 •

edited

Loading