fix(VR): depthbuffer culling when upscaling

[alandtse]

Depends on #1888

Fix by @vrnord from
https://github.com/vrnord/skyrim-community-shaders-VR-DLSS/tree/vr-depth-upscale-taa-reorder

Summary by CodeRabbit

Release Notes

• Bug Fixes

  • Shadow map cascade settings are now properly validated and bounded to supported limits, preventing invalid configurations.

• New Features

  • Added depth buffer optimization for VR rendering, improving performance and visual quality in virtual reality mode.

• Style

  • Corrected minor formatting inconsistencies in settings descriptions.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

This PR adds VR depth-buffer upscaling functionality and refines shadowmap cascade initialization. It introduces a new depth upscaling shader with associated constant buffers and rendering paths, clamps shadowmap cascade values during initialization, removes obsolete VR depth-culling constraint logic, and updates display strings in settings.

Changes

Cohort / File(s)	Summary
Shadowmap Cascade Fixes src/EngineFixes/ShadowmapCascadeRasterizerFix.cpp, src/EngineFixes/ShadowmapCascadeRasterizerFix.h	Clamped numCascades initialization between 1 and maxCascades; removed trailing spaces from INI setting display strings.
Upscaling Feature Infrastructure src/Features/Upscaling.h, src/Features/Upscaling.cpp	Added VR depth upscaling support: introduced GetDepthUpscalePS() method, DepthUpscaleCB constant buffer struct, depthUpscaleCB member, enableVRDepthUpscale flag, and depthUpscalePS shader pointer. Implemented dedicated depth-upscale pass in UpscaleDepth() for VR with resource setup and rendering logic. Removed GetActiveConstraints() override and obsolete VR depth-culling constraint forcing.
VR Depth Upscaling Shader features/Upscaling/Shaders/Upscaling/DepthUpscalePS.hlsl	New shader implementing conservative depth upscaling via 2x2 neighborhood minDepth sampling with dedicated hidden-area-mask handling, per-eye SBS clipping, and conservative bias blending.

Sequence Diagram

sequenceDiagram
    participant Main as Rendering Pipeline
    participant Upscale as Upscaling Module
    participant GPU as GPU/DirectX
    participant Shader as DepthUpscalePS
    participant DepthBuf as Depth Buffer

    Main->>Upscale: PerformUpscaling(isVR, scale)
    alt VR and enableVRDepthUpscale and scale != 1.0
        Upscale->>Upscale: UpscaleDepth()
        Upscale->>Upscale: GetDepthUpscalePS() [lazy compile]
        Upscale->>GPU: Setup DepthUpscaleCB (SourceDim, Scale)
        Upscale->>GPU: Bind DepthLowRes SRV
        Upscale->>GPU: Set render target & UAV
        Upscale->>GPU: Invoke depth-upscale pass
        GPU->>Shader: Execute DepthUpscalePS main()
        Shader->>DepthBuf: Sample 2x2 neighborhood
        Shader->>Shader: Compute minDepth (conservative)
        Shader->>GPU: Output upscaled depth (SV_Depth)
        GPU->>DepthBuf: Write upscaled depth
    end
    Upscale->>Main: Continue main upscaling pass

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Possibly related PRs

• feat(VR): enable upscaling #1507: Directly related to VR depth upscaling enablement and depth buffer management in the Upscaling feature.
• fix(upscaling): replace NIS with RCAS for DLSS #1620: Modifies the Upscaling feature with overlapping resource and member management changes.
• fix(VR): fix Shadowmap Cascade Rasterizer #1888: Modifies shadowmap cascade initialization logic in ShadowmapCascadeRasterizerFix alongside this PR's cascade clamping changes.

Suggested reviewers

• davo0411

Poem

🐰 A rabbit hops through VR's misty air,
Where depth is upscaled with tender care—
Conservative samples, a 2×2 dance,
Conservative bias, a forward glance!
Cascades now clamped, no more overflow,
Skyrim shades deeper, a VR glow! ✨

🚥 Pre-merge checks | ✅ 3 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 10.53% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'fix(VR): depthbuffer culling when upscaling' directly and clearly describes the primary change: fixing depth buffer culling behavior during VR upscaling operations.
Merge Conflict Detection	✅ Passed	✅ No merge conflicts detected when merging into dev

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

Using provided base ref: d5aee0a
Using base ref: d5aee0a
Base commit date: 2026-02-16T20:18:37+10:00 (Monday, February 16, 2026 08:18 PM)
No actionable suggestions for changed features.

[alandtse]

Looks like the upscale is a little off:
Before:

After:

[github-actions]

✅ A pre-release build is available for this PR:
Download

[alandtse]

Ok, looks like issues when submerged exist.

[coderabbitai]

🧹 Nitpick comments (4)

src/Features/Upscaling.h (1)

112-126: VR depth upscale scaffolding looks well-structured.

The DepthUpscaleCB layout matches the HLSL cbuffer declaration, and the new members follow the existing patterns in this class (e.g., jitterCB, upscalingDataCB).

One minor note: enableVRDepthUpscale defaults to true but doesn't appear to be exposed in DrawSettings() or serialized in SaveSettings/LoadSettings. Consider adding a UI toggle and persistence so users can disable this if it causes issues.

src/Features/Upscaling.cpp (1)

1609-1671: Missing render-state cleanup after VR depth-only pass.

The VR depth pass sets OMSetDepthStencilState, RSSetState, VSSetShader, PSSetShader, PSSetSamplers, and PSSetConstantBuffers but only unbinds SRVs at lines 1667-1668. The subsequent code block at line 1673 re-sets most of these, so it likely works in practice, but if resolutionScale.x == 1.0f somehow evaluates differently between the two blocks (or future code changes the flow), stale state could leak.

At minimum, consider unbinding the render target / DSV:

 		ID3D11ShaderResourceView* nullSRVs[1] = { nullptr };
 		context->PSSetShaderResources(0, 1, nullSRVs);
+		context->OMSetRenderTargets(0, nullptr, nullptr);

features/Upscaling/Shaders/Upscaling/DepthUpscalePS.hlsl (2)

59-63: SBS UV clamping is a good defense but doesn't fully protect GatherRed with a non-CLAMP sampler.

The clamp() on lines 62-63 correctly constrains the center sample point, but GatherRed fetches a 2×2 footprint that can extend one texel beyond the clamped coordinate when the sampler addressing is WRAP. This is complementary to the C++ fix (switching to a CLAMP sampler) — with CLAMP addressing, this UV clamping becomes redundant but harmless as a defense-in-depth measure.

---

65-77: Consider filtering out zero-depth samples from the minDepth computation instead of using a binary branch.

Currently, if any sample in the quad is 0 (HMD mask), the shader falls back entirely to point sampling. A more robust approach would filter zero-depth samples from the min() so valid depth neighbors still contribute to conservative culling near mask boundaries:

Suggested improvement

 	float4 depthQuad = DepthLowRes.GatherRed(LinearSampler, uv);
-	float minDepth = min(min(depthQuad.x, depthQuad.y), min(depthQuad.z, depthQuad.w));
-
-	// HMD hidden area mask: depth == 0 in reversed-Z.
-	// If ANY sample in the 2x2 quad is 0, we're at or near the mask boundary.
-	// Use point sampling only to avoid bilinear blending with mask pixels.
-	if (minDepth == 0.0) {
-		psout.Depth = DepthLowRes.Load(int3(texel, 0));
-		return psout;
-	}
-
-	// All four neighbors are valid depth. Blend point-sampled depth toward
-	// the conservative minimum for safe culling.
-	float pointDepth = DepthLowRes.Load(int3(texel, 0));
+	// Filter out HMD mask (depth == 0) from min computation
+	float minDepth = 1.0;  // Start at far plane (reversed-Z)
+	[unroll] for (int i = 0; i < 4; i++) {
+		if (depthQuad[i] > 0.0)
+			minDepth = min(minDepth, depthQuad[i]);
+	}
+
+	float pointDepth = DepthLowRes.Load(int3(texel, 0));
+
+	// If point sample itself is mask, output 0 (mask)
+	if (pointDepth == 0.0) {
+		psout.Depth = 0.0;
+		return psout;
+	}

This was also recommended in the PR discussion as a complementary defense-in-depth measure.

[alandtse]

I suspect this was actually unnecessary as we do upscale the depth buffer. The issue underwater is caused by the depthbuffer being upscaled twice. I'll likely keep this open for a few more days and close it out.

[alandtse]

Closing in favor of #1858