Final Fantasy XIV: VK_ERROR_DEVICE_LOST

[konomikitten]

There was a patch 16a51f3 to help reduce issues with FFXIV and Nvidia Drivers 450.66 and possibly above but this just seems to have made the issue occur less. After about 12 hours strait of having the FFXIV client running I once again got the dreaded VK_ERROR_DEVICE_LOST. I see other games in the issues list are having this problem too with the same driver versions. I'm starting to think Nvidia just messed up version 450.66. I can confirm I never got this error with 440.100 so I am pretty sure this is a Nvidia driver issue.

Software information

Final Fantasy XIV

System information

• GPU: GeForce GTX 960
• Driver: 450.66
• Wine version: lutris-5.7-10-x86_64
• DXVK version: 1.7.2

Log files

• d3d11.log: ffxiv_dx11_d3d11.log

[konomikitten]

I've patched and downgraded to 440.100 but with Kernel 5.8. I'll try to find a moment to leave the game running for at least 24 hours and see if I can replicate this bug. If I can't get this to happen again I'm really of the strong opinion Nvidia has introduced a bug into their drivers between the 44X and 45X series. Hopefully this information is useful for the dxvk devs.

[doitsujin]

We're aware of this but there's not an awful lot we can do about it on the DXVK side of things at this point.

[konomikitten]

So I tested 440.100 for 3 days (never restarted the client) and 0 issues 0 crashes. So if people want to continue using DXVK they should hold onto 440.100 as long as possible.

[sschroe]

Is 450.66 the first version where this issue was introduced? The Nvidia website also has 450.57 and 450.51. Knowing the exact version might be helpful for a bug report.

[konomikitten]

450.66 was the first version to have this issue from what I tested. Thankfully I've tested the latest long lived version and this issue appears to be fixed. I ran the tests again for 3 days continuous game client usage. I experienced no errors with the following:

Software information

Final Fantasy XIV

System information

• GPU: GeForce GTX 960
• Driver: 450.80.02
• Wine version: lutris-5.7-10-x86_64
• DXVK version: 1.7.2

Apitrace file(s)

None

Log files

• d3d9.log: None
• d3d11.log: ffxiv_dx11_d3d11.log
• dxgi.log: ffxiv_dx11_dxgi.log

I am unsure if the current short lived version (455.38 as of this post) has been fixed.

Despite all this testing I ended up with another crash on 450.80.02. Guess we wait more sadly.

[sschroe]

Hmm that's interesting because 450.80.02 does not mention any fixes but only support for some new GPUs. Though i have no idea how consistent nvidia it with their changelog. I'm pretty sure i saw the issue pop up on 455.38 when testing something unrelated though. But that may have been triggered by something else. Going to keep an eye out for it if it pops up again.

[K0bin]

Nvidia is aware of the problem and looking into it.

[arabcian]

Nvidia is aware of the problem and looking into it.

Do you have a link about it?

[khachbe]

Any updates on this issue?

[doitsujin]

That's generally not a very useful question to ask; if there was then someone would likely have posted it already. Unless Nvidia figure out whatever it is that's causing their driver to crash, this will not get fixed.

[astraldawn]

Occurs on 460.27.4 as well

[konomikitten]

455.45.01 also broken, this really makes me want to switch from nvidia to amd...

[konomikitten]

A possible fix but this is completely anecdotal. Try either clearing out your ~/.nv/GLCache/ or doing export __GL_SHADER_DISK_CACHE=0 with whatever way you're using wine. I haven't had a crash in awhile and this is the only thing I can think that I did lately that might affect DXVK.

[konomikitten]

There's now a post over at the Nvidia forums about this issue but I don't really have a high confidence in Nvidia fixing this (at least not with the replies I'm seeing on the post from them): Vk_error_device_lost in many game titles.

[astraldawn]

460.32.03 broken as well

[konomikitten]

Just to let everyone know I haven't had any issues since using export __GL_SHADER_DISK_CACHE=0. Currently on 460.32.03. Can others here test that?

Okay I had another crash so that doesn't work either, at this point I give up. People seeing this bug report need to go to the thread posted on the nvidia forums and actually post or this is never going to get fixed.

So please go add your voice to Vk_error_device_lost in many game titles.

[konomikitten]

Still broken 460.39.0.

ffxiv_dx11_d3d11.log
ffxiv_dx11_dxgi.log

[arabcian]

Just to let everyone know I haven't had any issues since using export __GL_SHADER_DISK_CACHE=0. Currently on 460.32.03. Can others here test that?

Okay I had another crash so that doesn't work either, at this point I give up. People seeing this bug report need to go to the thread posted on the nvidia forums and actually post or this is never going to get fixed.

So please go add your voice to Vk_error_device_lost in many game titles.

Hello im facing the same bug for about 6 months. I tried many blind workarounds to no avail lastly ive installed lutris added wine 6.0 and built my kernel and out of tree modules with clang. I couldnt reproduce it for 2 weeks still so early to talk but i just wonder if there are any changes in the dxvk code that may fix that or my blind tries finally gave me a solution. And i built nvidia-drivers with dynamic libraries. I hope this helps, I use Gentoo Linux where those workarounds are very easy to apply.

[konomikitten]

It's fun when you find out mpv also has the same problem with nvidia drivers: mpv-player/mpv#8360

[arabcian]

I still havent got any vk_error_device_lost yet with my shot but i sure ill get it at some time. Did you read the driver recommendation there? Please test one of those drivers and report here. Once i get that error again ill try one of those drivers too. Now no errors yet for 2 weeks.

January 27th, 2021 - Windows 457.88, Linux 455.50.04
Fixed a bug in a stencil-buffer optimization that could occasionally result in VK_ERROR_DEVICE_LOST

[konomikitten]

@arabcian my results are posted throughout this thread, multiple people are still having the issue and my driver version is far higher than yours sitting at 460.39.0 so I am pretty sure the fix your listed has nothing to do with the problem as was also mentioned on the mpv bug report where they don't make use of stencil-buffers either. I also clocked up 3 days of continuous play time only to have the bug randomly come back when I thought it was fixed.

[SveSop]

@konomikitten
Driver 460.39.0 is not necessarily "newer" than 455.50.04. The 455.50.xx driver series is of the "vulkan beta" series, and contains a lot more "vulkan fixes" than the "release driver" from the 460.xx series.

To simplify: 455.50.04 can contain a vulkan fix, the 460.39 driver does NOT contain since they are from two different branches.

When it comes to iffy nVidia random crashes + DXVK - Its always best to try the latest "vulkan beta" driver branch from here: https://developer.nvidia.com/vulkan-driver

[konomikitten]

@SveSop I stand corrected, that's some confusing version numbering.

[konomikitten]

@arabcian how many hours of FFXIV have you played to test for the crash by the way?

[konomikitten]

Just to let everyone know I am currently testing 455.50.04 as @arabcian recommended, it's looking good so far but considering the difficult nature of replicating this bug I'm going to be testing for a few weeks or more to make sure it is in fact fixed. If anyone else could test too that would be helpful.

[konomikitten]

Still broken. 455.50.04

ffxiv_dx11_d3d11.log
ffxiv_dx11_dxgi.log

[arabcian]

@arabcian how many hours of FFXIV have you played to test for the crash by the way?

I couldnt test it yet with FF since dont have the game but i was having same error with thief 2014 and World of Warcraft + Skyrim + Fallout 4,somehow bug is not around for a month. Still having mpd bug tho but im not sure if both are related.

[konomikitten]

Currently testing 455.50.07...

[konomikitten]

That's probably not the same problem. Make a separate issue.

Are you sure about that? My logs from 455.50.10 look a lot like @Retardium's...

Mine:

err:   DxvkSubmissionQueue: Failed to sync fence: VK_ERROR_DEVICE_LOST
err:   DxvkDevice: waitForIdle: Operation failed
err:   DxvkSubmissionQueue: Failed to sync fence: VK_ERROR_DEVICE_LOST
err:   DxvkDevice: waitForIdle: Operation failed
err:   DxvkSubmissionQueue: Failed to sync fence: VK_ERROR_DEVICE_LOST
err:   DxvkDevice: waitForIdle: Operation failed
err:   DxvkSubmissionQueue: Failed to sync fence: VK_ERROR_DEVICE_LOST
err:   DxvkDevice: waitForIdle: Operation failed
err:   DxvkSubmissionQueue: Command submission failed: VK_ERROR_DEVICE_LOST

Theirs:

warn:  D3D11Device::CreateShaderModule: Class linkage not supported
warn:  D3D11Device::CreateShaderModule: Class linkage not supported
warn:  D3D11Device::CreateShaderModule: Class linkage not supported
warn:  D3D11Device::CreateShaderModule: Class linkage not supported
warn:  D3D11Device::CreateShaderModule: Class linkage not supported
warn:  D3D11Device::CreateShaderModule: Class linkage not supported
warn:  D3D11DXGIDevice::QueryInterface: Unknown interface query
warn:  6543dbb6-1b48-42f5-ab82-e97ec74326f6
warn:  D3D11Device::CreateShaderModule: Class linkage not supported
warn:  D3D11Device::CreateShaderModule: Class linkage not supported
warn:  D3D11Device::CreateShaderModule: Class linkage not supported
warn:  D3D11Device::CreateShaderModule: Class linkage not supported
err:   DxvkSubmissionQueue: Failed to sync fence: VK_ERROR_DEVICE_LOST
err:   DxvkDevice: waitForIdle: Operation failed
err:   DxvkSubmissionQueue: Failed to sync fence: VK_ERROR_DEVICE_LOST
err:   DxvkDevice: waitForIdle: Operation failed
err:   DxvkSubmissionQueue: Failed to sync fence: VK_ERROR_DEVICE_LOST
err:   DxvkSubmissionQueue: Command submission failed: VK_ERROR_DEVICE_LOST
err:   DxvkDevice: waitForIdle: Operation failed

Edit: By the way two weeks on Kernel 5.4.109,110,111 and Nvidia 440.100 and 0 crashes, I am pretty sure I'm going to get to a month with 0 crashes as well. Really not liking Nvidia as usual.

[wasteoinc]

I also get the same issue on panzer Corps 2 on my gtx770 with both 450 and 460.

err:   DxvkSubmissionQueue: Failed to sync fence: VK_ERROR_DEVICE_LOST
err:   DxvkDevice: waitForIdle: Operation failed
err:   DxvkSubmissionQueue: Failed to sync fence: VK_ERROR_DEVICE_LOST
err:   DxvkDevice: waitForIdle: Operation failed
err:   DxvkSubmissionQueue: Failed to sync fence: VK_ERROR_DEVICE_LOST
err:   DxvkDevice: waitForIdle: Operation failed
err:   DxvkSubmissionQueue: Failed to sync fence: VK_ERROR_DEVICE_LOST
err:   DxvkDevice: waitForIdle: Operation failed

unfortunately 5.4.72 and 440.100 doesn't seem to help I get the same error

[konomikitten]

440.100 doesn't seem to help I get the same error

Are you absolutely sure you downgraded to 440.100?

[wasteoinc]

440.100 doesn't seem to help I get the same error

Are you absolutely sure you downgraded to 440.100?

Unfortunately yes, else I would be a happy gamer. But maybe the problem lies with the specific game (panzer corps 2) . I will try some other games next week and see if I have the same issue

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 440.100      Driver Version: 440.100      CUDA Version: 10.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 770     Off  | 00000000:07:00.0 N/A |                  N/A |
| 30%   49C    P8    N/A /  N/A |    184MiB /  1994MiB |     N/A      Default |
+-------------------------------+----------------------+----------------------+

[konomikitten]

Unfortunately yes, else I would be a happy gamer.

I guess @K0bin was right then sorry about that K0bin.

[wasteoinc]

Unfortunately yes, else I would be a happy gamer.

I guess @K0bin was right then sorry about that K0bin.

Well I will try to do a full purge of nvidia and everything and install 440.100 from command line so I can more properly track what's going on because I found so many different versions of nvidia packages in my installation.

give me a bit more time..

ehm do you have a guide on how to install 440.100 ? even though the drivers installs in command line pretty well now that I purged everything when I boot it loads the mesa driver. (shy)
I try to install the run file and I get to the libglvnd error. And the PPA doesn't seem to include 440 anymore. So I get now to this stage
NVRM: API mismatch: the client has the version 440.100, but
NVRM: this kernel module has the version 460.67. Please
NVRM: make sure that this kernel module and all NVIDIA driver
NVRM: components have the same version.

[wasteoinc]

2 hours of clusterfuck and driver parts here and there and I managed to load a proper clean 440.100 installation (no 32bit support and no DKMS though) . I managed to actually play for more time than I ever managed on this game (like an hour) but it eventually crashed to our usual issue. I have to go to work now I will try some more with frostpunk tonight to see if it a reaccuring thing.

[konomikitten]

I've downgraded all the way to 440.100 again and I'll be tested that for a good month or so.

So it's been a month now and no crashes, I'm just going to stick with this kernel and driver version. I've pretty much given up on trying to update the Nvidia Driver and not have this issue occur.

Software information

Final Fantasy XIV

System information

• GPU: GeForce GTX 960
• Kernel: 5.4.115
• Driver: 440.100
• Wine version: lutris-6.4-x86_64
• DXVK version: 1.8.1

Log files

• d3d11.log: ffxiv_dx11_d3d11.log

[sfjuocekr]

I used to have a bunch of crashes with the 46x series until they "fixed" 465 and 470 works as well.

But you should be using: 455.50.19

I can highly recommend purging the 4xx drivers entirely, make sure you clean up the dkms folder as well and then do a manual install. You will probably lose video if you try this, so do it from a SSH console.

This driver gives by far the best performance on my GTX970 and zero crashes thus far.

[arabcian]

I upgraded my laptop now i have RTX 3060 mobile GPU and i can confirm errors are no more with any driver ive tried. This error is called something like xid31 by nvidia and latest 470 drivers have a fix for it.

[nzbtuxnews]

Just found out about this issue and reported it on ticket 2253

I have drivers 470.57.2 and this issue is NOT fixed.

[K0bin]

That's most likely an unrelated issue.

DEVICE_LOST just means the driver encountered an error for one reason or another.

[neskweek]

Still Broken with 470.63.01

[konomikitten]

@doitsujin considering people are still reporting this is broken, and #1963 made the major version check for the nvidia work around 465, shouldn't this just default to being back on for all Nvidia versions? As far as I know this bug was the reason the work around existed in the first place. It doesn't seem like Nvidia actually fixed it.

[doitsujin]

I'm not convinced that the issues people are reporting are actually caused by the same issue. There's still a problem somewhere regarding high HVV use but I've never seen it crash since.

[konomikitten]

There's still a problem somewhere regarding high HVV use but I've never seen it crash since.

I'm currently testing 470.74 if I still manage to end up making the game crash (could take a month or more) but I've still never in at least a year of playing FFXIV had 440.100 crash, where does that ultimately put things? Is this just a rare bug that will never be fixed? Do you have any theories on this at all? I would've just escaped this by switching to an AMD card but with the whole market the way it is it hasn't been an option.

Do you have any thoughts on this at all? It's driving me nuts.

[SveSop]

Interesting note on the 470.74 tho:

Fixed a regression which resulted in very-high system memory usage for Direct3D 12 games when run through vkd3d-proton.

Could be useful for DXVK too maybe? (Ie. high-memory related issue).

That said, nVidia hardware CAN sometimes be throwing those random XID errors when different manufacturers are pushing the limits on their "factory clocks" (Eg. Asus/MSI++ with their "OC" models and whatnot). Could be worth TRYING to underclock or powerlimit the gpu just to see.

You can check your powerlimit with nvidia-smi Then if it has like 215W (like my RTX2070 has), then you could try to use
sudo nvidia-smi -pl 200
Just mentioning this because i had a nVidia (260Ti card back when) that was really unstable - but it was one of those "OC Twin Frozr Whatever" cards, and ended up just downclocking it a tad, and issue went away :) Loads of factors playing in here - especially since XID(DEVICE LOST) errors MAY not be driver/dxvk related at all.

[doitsujin]

Could be useful for DXVK too maybe? (Ie. high-memory related issue).

No, the issue was specifically caused by the way vkd3d-proton uses Vulkan pipeline caches. DXVK is not affected by this in any way.

[konomikitten]

Honestly at this point I'll willing to concede this might just be a game bug. I've looked through the tracker and see other games that will crash with an in game bug but still have the DXVK log producing VK_ERROR_DEVICE_LOST. There are multiple numerous reports from Windows users on the FFXIV forums about DX11 crashes. I also play with my significant other who has had both the DX11 crash and sometimes a freeze while playing the game and they use Windows and AMD.

I think the biggest problem here is VK_ERROR_DEVICE_LOST is just too much of a generic error, is there anyway DXVK can give a better error code?

I think it's probably time to close this bug, I personally after all this time don't think this is a Nvidia or DXVK bug. Thoughts?

[sschroe]

I don't think that the crashes on Windows are related to this. I have several friends using Nvidia cards (including RTX 3xxx) on Windows that afk all day in-game and never mentioned anything about the game crashing. And if this was actually a widespread issue there'd be a lot more noise about it with the current login queues. The people on windows having crashes more likely is borked installations, hardware or overclocks.

And even on Linux only newer Nvidia drivers are affected by it reproducibly, older ones and AMD cards run rock solid. Since switching to a 6900 XT I haven't had a single crash in the game. So from my point of view this still looks like a bug in Nvidias driver on Linux.

As far I know Nvidia devs also browse through these issues once in a while so keeping the issue open might make sense for the small chance of them looking into it one day, but more practically this would belong on the Nvidia forums or whatever they use for bug reports.

[konomikitten]

And if this was actually a widespread issue there'd be a lot more noise about it with the current login queues. The people on windows having crashes more likely is borked installations, hardware or overclocks.

There is an extensive thread over on the Final Fantasy Forums though: https://forum.square-enix.com/ffxiv/threads/448687-Directx11-error

Users have the following hardware in that thread:

AMD Ryzen 6600XT
AMD Ryzen 7 5800X

AMD Ryzen 7 5800X 8-Core Processor
AMD Radeon RX 6600 XT

AMD Ryzen 5 3600 6-Core Processor
AMD Radeon RX 5700 XT

AMD Ryzen 5 3600 6-Core Processor
Radeon RX 580 Series

Intel(R) Core(TM) i7-8700 CPU
NVIDIA GeForce RTX 2070

AMD Ryzen 7 5700G with Radeon Graphics
NVIDIA GeForce GTX 1080

AMD Ryzen 5950x
AMD Radeon 6900x

As one user on the thread states:

Nothing has fixed this issue. Sometimes we can play all day with no problems, the next day the DirectX 11 11000002 error comes up over and over. If you lookup that error number, you'll see MILLIONS of posts online (not just on these forums) about this FF14 issue. This is not an obscure thing, A LOT OF PEOPLE have this problem.

And another user said it's been there since Stormblood:

A large part of the community have been waiting for an answer regarding this issue since Stormblood release, so I'm not sure if we are going to get one anytime soon if we didn't get one so far.

[ZeroPointEnergy]

Apart from the VK_ERROR_DEVICE_LOST I also randomly get complete nvidia driver crashes with this game. I first thought my GPU is probably defect, but Xid 16 means “Display engine hung” and “Driver Error” according to nvidias own list and I do play other games, sometimes for hours and I have never seen such a crash outside of FFXIV.

This is with a NVIDIA GeForce GTX TITAN X card.

[11303.224211] NVRM: GPU at PCI:0000:01:00: GPU-0b4cda80-07b2-13fb-9c08-f03bf0381c28
[11303.224213] NVRM: Xid (PCI:0000:01:00): 16, pid=0, Head 00000000 Count 000b1ef6
[11311.534091] NVRM: Xid (PCI:0000:01:00): 16, pid=16, Head 00000000 Count 000b1ef7
[11319.853967] NVRM: Xid (PCI:0000:01:00): 16, pid=6226, Head 00000000 Count 000b1ef8
[11328.183850] NVRM: Xid (PCI:0000:01:00): 16, pid=0, Head 00000000 Count 000b1ef9
[11336.503749] NVRM: Xid (PCI:0000:01:00): 16, pid=0, Head 00000000 Count 000b1efa
[11344.813634] NVRM: Xid (PCI:0000:01:00): 16, pid=0, Head 00000000 Count 000b1efb
[11353.133512] NVRM: Xid (PCI:0000:01:00): 16, pid=0, Head 00000000 Count 000b1efc
[11361.463408] NVRM: Xid (PCI:0000:01:00): 16, pid=0, Head 00000000 Count 000b1efd
[11369.783297] NVRM: Xid (PCI:0000:01:00): 16, pid=0, Head 00000000 Count 000b1efe
[11378.093175] NVRM: Xid (PCI:0000:01:00): 16, pid=0, Head 00000000 Count 000b1eff
[11386.413068] NVRM: Xid (PCI:0000:01:00): 16, pid=0, Head 00000000 Count 000b1f00
[11394.732946] NVRM: Xid (PCI:0000:01:00): 16, pid=0, Head 00000000 Count 000b1f01
[11403.062837] NVRM: Xid (PCI:0000:01:00): 16, pid=0, Head 00000000 Count 000b1f02
[11411.372721] NVRM: Xid (PCI:0000:01:00): 16, pid=0, Head 00000000 Count 000b1f03
[11419.692602] NVRM: Xid (PCI:0000:01:00): 16, pid=0, Head 00000000 Count 000b1f04
[11428.022469] NVRM: Xid (PCI:0000:01:00): 16, pid=0, Head 00000000 Count 000b1f05
[11463.772567] nvidia-modeset: WARNING: GPU:0: Timeout while waiting for idle.
[11465.772637] nvidia-modeset: ERROR: GPU:0: Idling display engine timed out: 0x0000957d:0:0:428

Made a post on their forums, sent a bug report to the "support" address, never even got an answer back.

Also should mention that the driver crash is 100% not a DXVK bug, since it also happens with pure wine (only tested once, because the performance is horrible). I'm not sure if this is related at all to the original bug here, but I just wanted to add that information, in the hope it may help. If this confuses things more, please just disregard.

[konomikitten]

I first thought my GPU is probably defect, but Xid 16 means “Display engine hung” and “Driver Error” according to nvidias own list and I do play other games, sometimes for hours and I have never seen such a crash outside of FFXIV.

And that's the problem I play other games outside of FFXIV and they never crash either, we have a bunch of windows users reporting the same thing on different GPUs and if you take a look at other bug reports like this one #2349 you'll see that bugs in games can definitely cause VK_ERROR_DEVICE_LOST and not just problems with Nvidia or DXVK.

As I said I've played this game a lot and so has my significant other I've watched their PC running Windows have all the same crashes I do on DXVK and Nvidia and they're using Windows and AMD.

I believe the bug is in the game itself for this reason, I also believe that depending on hardware/driver versions the bug can occur less or more frequently. I only wish I had the money to experiment with different GPUs and end this debate for once and all.

[Blisto91]

@konomikitten How is this behaving with the 515 drivers if you are still testing this from time to time.

[konomikitten]

@Blisto91 I switched from my GTX 960 to a RX 6600 about 4 months ago and I haven't had this crash since. My significant other switched from an RX 570 to a RX 6600XT and now gets this issue on Windows 10.

I still see frequent posts from Windows users on the Final Fantasy Forums about DirectX Error 11000002. As far as I am concerned this is a game bug that Square need to fix.

[astraldawn]

@konomikitten How is this behaving with the 515 drivers if you are still testing this from time to time.

No issues with 515.48.07 (as stable as 440.100)

[konomikitten]

@konomikitten How is this behaving with the 515 drivers if you are still testing this from time to time.

No issues with 515.48.07 (as stable as 440.100)

I no longer use a Nvidia Graphics card as per my previous post so I cannot answer that question.

[Blisto91]

I think they are stating that the 515 drivers are stable for them.