Nvidia's own documentation claims that if we return NVAPI_D3D12_RAYTRACING_OPACITY_MICROMAP_CAP_NONE from GetRaytracingCaps, then

The application must not attempt to use any OMM entrypoints or flags.

So, no, we can't stub it and have the sample working if it tries to call SetCreatePipelineStateOptions anyway because that sounds like a blatant violation of Nvapi docs. And actually implementing it would take much more than just some dxvk-nvapi changes, vkd3d-proton would have to support that as well. I have some WIP branches for that in my own forks (dxvk-nvapi, vkd3d-proton) but it doesn't work yet and will probably never be merged upstream unless there's a good reason for it, and right now there is none.

Is there any game that uses Nvapi OMMs out there? Because if not then there's no reason to spend more time on that other than having fun working on a toy project.

Hi,
thanks @Saancreed ..
seems didn't give much thought.. thanks for explaining..
anyway given you have given some time try on your own dxvk-nvapi and vkd3d-proton, just a question would you be interested to work on supporting the NVAPI D3D12 SER extension via the VK NV VK_NV_ray_tracing_invocation_reorder extension..
seems more "easy" to support being just a "hint" passed to RT pipelines from quick inspection..
also should be more useful in real world:
for example "Sackboy: A Big Adventure" supports that seems, and NV Path tracing SDK sample (in D3D12 mode) and perhaps Omniverse DX12(?)..
Portal RTX uses SER, I don't know if it's VK or DX12 since it uses DXVK but I believe it's in DX12 mode also?

You're welcome 🙂

anyway given you have given some time try on your own dxvk-nvapi and vkd3d-proton, just a question would you be interested to work on supporting the NVAPI D3D12 SER extension via the VK NV VK_NV_ray_tracing_invocation_reorder extension..
seems more "easy" to support being just a "hint" passed to RT pipelines from quick inspection..

It would be easier to support dxvk-nvapi and vkd3d-proton sides of the stack, but much harder to support in dxil-spirv which is where all the magic needs to happen. All we have to do is claim that SER is supported and have the application hand us DXIL blobs with NV extension opcodes. On the other hand, dxil-spirv would have to parse that and emit equivalent SPIR-V and I'm very much not knowledgeable enough to do that 😛

also should be more useful in real world:
for example "Sackboy: A Big Adventure" supports that seems, and NV Path tracing SDK sample (in D3D12 mode) and perhaps Omniverse DX12(?)..

I know that Cyberpunk 2077 also checks for SER support so there's a good chance it uses it when supported, but since we have some actual titles that could benefit from this, maybe it's a good enough reason to request support for it in vkd3d-proton's repo/issues? At best, one of the maintainers could look into implementing that and we could wire up nvapi side once that's done, at worst we would have an issue to keep track of such feature request until some Nvidia engineer contributes the code for it 😅

Portal RTX uses SER, I don't know if it's VK or DX12 since it uses DXVK but I believe it's in DX12 mode also?

There's no DX12 mode in Portal RTX, it ships with its own version of DXVK so it's effectively a native Vulkan application from Wine's point of view. It should already be able to take advantage of VK_NV_ray_tracing_invocation_reorder on any device that supports it, with no extra work needed in dxvk-nvapi, in fact it even prints in the logs:

info:  Device properties:
info:    Device name:     : NVIDIA GeForce RTX 4080 Laptop GPU
info:    Driver version   : 530.41.3
info:  Enabled device extensions:
…
info:    VK_EXT_opacity_micromap
…
info:    VK_NV_ray_tracing_invocation_reorder
…
info:  NVIDIA Ada architecture detected, setting default graphics settings to Ultra
info:  [RTX info] Opacity Micromap: supported
info:  [RTX info] Shader Execution Reordering: supported
info:  [RTX info] Shader Execution Reordering: enabled