fix: proton shadow taa jitter#1665
Conversation
📝 WalkthroughWalkthroughThe PR introduces runtime detection for DXVK/Wine environments and conditionally adjusts shadow denoising strategies. When detected, position-based hashing replaces frame-count-based randomness to reduce temporal artifacts in shadow sampling on non-native D3D11 platforms. Changes
Sequence DiagramsequenceDiagram
participant CPU as CPU / State.cpp
participant SharedMem as Shared Data Buffer
participant GPU as GPU / Utility.hlsl
CPU->>CPU: Inspect adapter description
alt DXVK Detected
CPU->>CPU: isRunningOnDXVK = true
else Native D3D11
CPU->>CPU: isRunningOnDXVK = false
end
CPU->>SharedMem: Populate IsNotNativeD3D11 flag
CPU->>GPU: Upload SharedData cbuffer
GPU->>GPU: GetPoissonDiskFilteredShadowVisibility()
alt IsNotNativeD3D11 == 1
GPU->>GPU: frameOffset = position-based hash
note over GPU: Spatial denoising (no temporal jitter)
else IsNotNativeD3D11 == 0
GPU->>GPU: frameOffset = frame count
note over GPU: Temporal denoising (native D3D11)
end
GPU->>GPU: Apply filtered shadow sampling
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested reviewers
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Automated formatting by clang-format, prettier, and other hooks. See https://pre-commit.ci for details.
davo0411
moved this from Priority Issues
to In review
in Community Shaders Development Tracker
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
package/Shaders/Common/SharedData.hlslipackage/Shaders/Utility.hlslsrc/State.cppsrc/State.h
🧰 Additional context used
📓 Path-based instructions (3)
**/*.{cpp,cxx,cc,c,h,hpp,hxx,hlsl,hlsli,fx,fxh,py}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Do not include TODO/FIXME placeholders; provide complete, working solutions
Files:
src/State.cppsrc/State.hpackage/Shaders/Utility.hlslpackage/Shaders/Common/SharedData.hlsli
src/**/*.{cpp,cxx,cc,h,hpp,hxx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
src/**/*.{cpp,cxx,cc,h,hpp,hxx}: Ensure SE/AE/VR runtime compatibility; use runtime detection patterns (e.g., REL::RelocateMember())
Include robust error handling and resource management with graceful degradation in the plugin code
Files:
src/State.cppsrc/State.h
**/*
⚙️ CodeRabbit configuration file
**/*: When reviewing PRs, please provide suggestions for:
Conventional Commit Titles (if not following https://www.conventionalcommits.org/ or
if the existing title does not describe the code changes):
Format: type(scope): description
Length: 50 characters limit for title, 72 for body
Style: lowercase description, no ending period
Examples:
- feat(vr): add cross-eye sampling
- fix(water): resolve flowmap bug
- docs: update shader documentation
Issue References (if PR fixes bugs or implements features):
Suggest adding appropriate GitHub keywords:Otherwise, use your standard review approach focusing on code quality.
Files:
src/State.cppsrc/State.hpackage/Shaders/Utility.hlslpackage/Shaders/Common/SharedData.hlsli
🧠 Learnings (9)
📓 Common learnings
Learnt from: alandtse
Repo: doodlum/skyrim-community-shaders PR: 0
File: :0-0
Timestamp: 2025-07-05T05:20:45.823Z
Learning: In the skyrim-community-shaders repository, file deletion error handling improvements that replace existence checks and try-catch blocks with std::filesystem::remove error-code-based approaches are considered bug fixes rather than refactoring, as they address inadequate error handling and misleading log messages.
Learnt from: jiayev
Repo: doodlum/skyrim-community-shaders PR: 0
File: :0-0
Timestamp: 2025-08-03T18:37:19.690Z
Learning: ISReflectionsRayTracing.hlsl and ISWorldMap.hlsl in the skyrim-community-shaders repository are image-space post-processing shaders that perform color sampling and blending operations that need proper linear color space handling for the linear lighting system. ISReflectionsRayTracing handles screen-space reflections and should use conditional Color::IrradianceToLinear/Gamma conversions similar to ISCompositeLensFlareVolumetricLighting.hlsl. ISWorldMap performs 7x7 color accumulation that should be done in linear space similar to the pattern used in ISSAOComposite.hlsl.
📚 Learning: 2025-06-08T11:25:14.536Z
Learnt from: alandtse
Repo: doodlum/skyrim-community-shaders PR: 577
File: src/Features/WetnessEffects.h:36-36
Timestamp: 2025-06-08T11:25:14.536Z
Learning: In the skyrim-community-shaders project, boolean flags in C++ structs that interface with HLSL shaders use `uint` type instead of `bool` for compatibility reasons. This ensures consistent 4-byte size, proper 16-byte alignment in constant buffers, and cross-platform compatibility when passing data between C++ and HLSL shaders.
Applied to files:
src/State.hpackage/Shaders/Utility.hlslpackage/Shaders/Common/SharedData.hlsli
📚 Learning: 2025-06-08T11:25:14.536Z
Learnt from: alandtse
Repo: doodlum/skyrim-community-shaders PR: 577
File: src/Features/WetnessEffects.h:36-36
Timestamp: 2025-06-08T11:25:14.536Z
Learning: In the skyrim-community-shaders project, boolean flags in C++ structs that interface with HLSL shaders use `uint` type instead of `bool` for compatibility reasons. This ensures consistent size, alignment, and cross-platform compatibility when passing data to shader constant buffers.
Applied to files:
src/State.hpackage/Shaders/Utility.hlslpackage/Shaders/Common/SharedData.hlsli
📚 Learning: 2025-08-03T18:37:19.690Z
Learnt from: jiayev
Repo: doodlum/skyrim-community-shaders PR: 0
File: :0-0
Timestamp: 2025-08-03T18:37:19.690Z
Learning: ISReflectionsRayTracing.hlsl and ISWorldMap.hlsl in the skyrim-community-shaders repository are image-space post-processing shaders that perform color sampling and blending operations that need proper linear color space handling for the linear lighting system. ISReflectionsRayTracing handles screen-space reflections and should use conditional Color::IrradianceToLinear/Gamma conversions similar to ISCompositeLensFlareVolumetricLighting.hlsl. ISWorldMap performs 7x7 color accumulation that should be done in linear space similar to the pattern used in ISSAOComposite.hlsl.
Applied to files:
package/Shaders/Utility.hlsl
📚 Learning: 2025-08-17T18:37:35.839Z
Learnt from: CR
Repo: doodlum/skyrim-community-shaders PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-08-17T18:37:35.839Z
Learning: Applies to features/*/Shaders/**/*.{hlsl,hlsli,fx,fxh} : Avoid GPU register/buffer conflicts in HLSL; verify register usage (e.g., with hlslkit buffer scanning)
Applied to files:
package/Shaders/Utility.hlslpackage/Shaders/Common/SharedData.hlsli
📚 Learning: 2025-10-02T14:20:33.454Z
Learnt from: ThePagi
Repo: doodlum/skyrim-community-shaders PR: 1369
File: src/Features/SnowCover.cpp:515-515
Timestamp: 2025-10-02T14:20:33.454Z
Learning: In the Community Shaders codebase (skyrim-community-shaders repository), hardcoded shader resource slot numbers are used directly in code rather than being defined as named constants. This is the established convention and should not be flagged as an issue.
Applied to files:
package/Shaders/Utility.hlslpackage/Shaders/Common/SharedData.hlsli
📚 Learning: 2025-07-01T18:01:07.079Z
Learnt from: alandtse
Repo: doodlum/skyrim-community-shaders PR: 0
File: :0-0
Timestamp: 2025-07-01T18:01:07.079Z
Learning: In the skyrim-community-shaders project, simple scalar constants in HLSL shaders use #define (e.g., #define NTHREADS 128), while more complex constants use static const within namespaces (e.g., Math namespace in Math.hlsli). For epsilon standardization, #define is the appropriate choice since epsilon values are simple scalar constants.
Applied to files:
package/Shaders/Utility.hlsl
📚 Learning: 2025-07-05T05:20:45.823Z
Learnt from: alandtse
Repo: doodlum/skyrim-community-shaders PR: 0
File: :0-0
Timestamp: 2025-07-05T05:20:45.823Z
Learning: In the skyrim-community-shaders repository, file deletion error handling improvements that replace existence checks and try-catch blocks with std::filesystem::remove error-code-based approaches are considered bug fixes rather than refactoring, as they address inadequate error handling and misleading log messages.
Applied to files:
package/Shaders/Utility.hlsl
📚 Learning: 2025-08-05T17:40:44.828Z
Learnt from: ThePagi
Repo: doodlum/skyrim-community-shaders PR: 1369
File: package/Shaders/Lighting.hlsl:0-0
Timestamp: 2025-08-05T17:40:44.828Z
Learning: In the skyrim-community-shaders repository, ultra trees (object LOD trees) are detected using a compound shader define condition `defined(DO_ALPHA_TEST) && defined(LOD_BLENDING) && defined(RIM_LIGHTING) && defined(SOFT_LIGHTING)` because "they have no define" according to the comment. The `ExtraFlags::IsTree` flag is used for different tree handling (AO removal in skylighting) and may not apply to ultra trees specifically. Before replacing the compound condition with `IsTree`, verification is needed to ensure the flag covers ultra trees.
Applied to files:
package/Shaders/Common/SharedData.hlsli
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: Validate shader compilation (Flatrim, .github/configs/shader-validation.yaml)
- GitHub Check: Validate shader compilation (VR, .github/configs/shader-validation-vr.yaml)
- GitHub Check: Build plugin and addons
- GitHub Check: Run Shader Unit Tests
🔇 Additional comments (6)
package/Shaders/Common/SharedData.hlsli (1)
26-26: LGTM - Field addition aligns with C++ struct.The
uint IsNotNativeD3D11field correctly replaces the previous padding and usesuinttype for proper C++/HLSL constant buffer compatibility. Based on learnings, this project usesuintinstead ofboolfor shader interface compatibility.package/Shaders/Utility.hlsl (2)
378-386: LGTM - Position-based offset for DXVK shadow stability.The spatial hashing approach using
murmur3with quantized world position provides stable shadow sampling patterns that won't flicker with TAA jitter on DXVK. The conditional correctly preserves the original frame-based behavior on native D3D11.The quantization factor of 8.0 means the hash changes every 0.125 world units. This could potentially cause visible banding patterns in shadow penumbras at certain viewing angles. Consider testing with different quantization values if artifacts appear.
627-635: LGTM - Consistent spatial noise for DXVK path.The noise generation mirrors the shadow sampling approach, providing stable per-pixel noise based on world position rather than frame count when running under DXVK. This prevents the shimmering artifacts caused by TAA jitter in the translation layer.
src/State.cpp (1)
791-792: LGTM - Correct propagation of DXVK flag to shader.The boolean-to-uint conversion follows project conventions for shader constant buffer compatibility. Based on learnings, using
uintinstead ofboolensures consistent 4-byte size and proper alignment.src/State.h (2)
48-48: LGTM - Runtime detection flag.The
isRunningOnDXVKboolean flag provides clean runtime state tracking for the DXVK detection, populated during adapter description parsing.
207-207: LGTM - Shader-compatible flag replaces padding.The
uint IsNotNativeD3D11field correctly replaces the previousfloat pad0with a meaningful flag while maintaining the struct's 16-byte alignment. Usinguintensures proper C++/HLSL constant buffer compatibility per project conventions.
|
|
||
| // Detect Linux/Proton/DXVK by checking adapter description | ||
| std::string lowerDesc = adapterDescription; | ||
| std::transform(lowerDesc.begin(), lowerDesc.end(), lowerDesc.begin(), ::tolower); | ||
| isRunningOnDXVK = (lowerDesc.find("dxvk") != std::string::npos) || | ||
| (lowerDesc.find("wine") != std::string::npos) || | ||
| (lowerDesc.find("llvmpipe") != std::string::npos); | ||
| if (isRunningOnDXVK) { | ||
| logger::info("Detected Linux/Proton/DXVK environment - using spatial shadow denoising"); | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
DXVK adapter description string D3D11 GetAdapterDesc
💡 Result:
Short answer
- DXVK fills DXGI_ADAPTER_DESC.Description with the Vulkan physical device name (VkPhysicalDeviceProperties::deviceName, converted to wide chars). VendorId/DeviceId come from VkPhysicalDeviceProperties.vendorID/deviceID and memory fields from the device memory info; DXGI structs otherwise follow DXGI_ADAPTER_DESC semantics. Sources: DXVK uses Vulkan physical-device properties when enumerating adapters, Vulkan exposes deviceName/vendorID/deviceID, and DXGI_ADAPTER_DESC defines the Description field. [1][2][3]
Sources
- DXVK project (repository / adapter handling / docs showing DXVK maps Vulkan physical devices to DXGI adapters). [1]
- Vulkan reference: VkPhysicalDeviceProperties (deviceName, vendorID, deviceID). [2]
- Microsoft: DXGI_ADAPTER_DESC / IDXGIAdapter::GetDesc (Description field semantics). [3]
🌐 Web query:
Wine DXVK llvmpipe adapter name identification
💡 Result:
Short answer — identify the llvmpipe adapter by checking Vulkan device names (vulkaninfo) or DXVK logs, then use that string with DXVK_FILTER_DEVICE_NAME to exclude/choose it.
How to do it
-
List Vulkan devices: run
vulkaninfo | grep -i deviceName
The output shows lines like "deviceName = llvmpipe (LLVM 15.0.7, 256 bits)" for software (llvmpipe) and your real GPU(s). Use the exact substring you want to match. [1][2] -
Or read DXVK startup logs: DXVK prints adapter names when enumerating devices, e.g. "Skipping Vulkan 1.0 adapter: llvmpipe (LLVM ...)" — inspect your DXVK stdout/log to see the same name. [2]
-
Force/select a device in DXVK:
export DXVK_FILTER_DEVICE_NAME="Your GPU device substring"
(DXVK matches substrings from the Vulkan device name; if the filter matches the wrong device or none, DXVK will fail to create a device — use the exact substring from vulkaninfo/DXVK logs). [3]
Notes
- If Vulkan lists only llvmpipe, the system is using Mesa software rendering (install or fix your GPU Vulkan ICD or set VK_ICD_FILENAMES to the GPU ICD JSON). [2]
- You can also select by UUID with DXVK_FILTER_DEVICE_UUID (vulkaninfo shows UUIDs). [3]
Sources
- Example of llvmpipe in Vulkan output / Mesa issues. [1]
- DXVK logging and examples showing llvmpipe in adapter list. [2]
- DXVK device-filter environment variables (DXVK_FILTER_DEVICE_NAME / UUID). [3]
References
[1] Mesa / llvmpipe example (vulkan/glxinfo output).
[2] DXVK/GitHub issues and community posts showing "llvmpipe" in DXVK logs.
[3] DXVK documentation / README (DXVK_FILTER_DEVICE_NAME and DXVK_FILTER_DEVICE_UUID).
🌐 Web query:
How to detect DXVK Wine Proton in Direct3D adapter description
💡 Result:
Short answer: you generally can't reliably detect DXVK/Proton/Wine just from the Direct3D adapter description alone. Instead check for DXVK-specific signs at runtime (logs, HUD, DLLs, env vars) or for mismatches between the reported adapter string and the underlying Vulkan driver. Sources: DXVK docs and project pages. [1][2][3][4]
Practical checks (ordered from most reliable to least):
- Look for DXVK logs or output
- DXVK prints logs to stderr when used under Wine/Proton (or to files when DXVK_LOG_PATH set). If you can capture process stderr or the game folder, presence of d3d9/d3d11/dxgi log messages proves DXVK. [2][3]
- Check for DXVK binaries / DLL overrides in the prefix/process
- If the process loads DXVK DLLs (d3d11.dll, dxgi.dll, d3d9.dll from dxvk) that’s definitive. Inspect loaded modules (Process Explorer / lsof / /proc//maps) or the Wine prefix to see installed DLLs. [2][3]
- Use DXVK_HUD or environment variables
- DXVK has a HUD (DXVK_HUD) that can show "version", "devinfo" and device strings. If enabling DXVK_HUD produces the overlay, DXVK is active. Also check for DXVK_/VKD3D_ env vars in the process environment. [1][4]
- Detect vendor/driver mismatch (heuristic only)
- Under DXVK the adapter description will often reflect the host Vulkan driver (Mesa, RADV, NVIDIA) rather than a Windows native driver. If the adapter string contains obvious Linux/Vulkan driver names (e.g., "RADV", "Mesa", "vulkan", "Intel Vulkan"), or mentions "Proton"/"Wine" in other outputs, that’s a hint but not definitive. This is fragile and can be spoofed. [1][4]
- For D3D12: check for vkd3d(-proton)
- D3D12 translation on Proton uses vkd3d(-proton) rather than DXVK; look for vkd3d logs, DLLs or HUD differences. [1][3]
Example quick commands (Linux/Wine host)
- Check loaded modules: lsof -p or grep dxgi /proc//maps (shows dxgi/d3d11 DLL path). (See point 2.) [2][3]
- Run with DXVK_HUD=devinfo,fps to force the overlay and verify. (See point 3.) [1][4]
- Inspect stderr or set DXVK_LOG_PATH to force log files and inspect them for "info: " lines. [2][3]
References
- DXVK official site / FAQ (HUD, usage). [1]
- DXVK README / setup and logging behavior (how it emits logs under Wine/Proton and DLLs). [2][3]
- HOWTO / Linux pages describing DXVK_HUD and devinfo behavior. [4]
If you want, tell me whether you need a small script (Windows/Wine) to perform these checks automatically and I’ll give one.
Detection keywords are unlikely to match adapter descriptions.
The DXVK adapter description is populated from the Vulkan physical device name (e.g., "llvmpipe (LLVM ...)", "RADV", "Intel Vulkan"), not from DXVK or Wine identifiers. Checking for "dxvk" or "wine" strings in the adapter description will not reliably detect DXVK/Proton environments—these translation layer names do not appear in the reported device names. The "llvmpipe" check is valid and will match software rendering, but it doesn't specifically identify DXVK or Wine.
For reliable detection of DXVK/Wine at runtime, consider checking for DXVK-specific signs: presence of DXVK logs, DLL overrides, environment variables (DXVK_*), or the DXVK HUD. Adapter string matching alone is insufficient for this purpose.
|
✅ A pre-release build is available for this PR: |
2 participants
should address #1479
replaces typical taa shadow accumulation with a worldspace hash function variant when dxvk, wine, llvmpipe are detected in the gpu adapter description.
this list might be wrong as i am unfamiliar with linux and just got common names.
untested
Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings.