fix(deferred): restore UAV bind on vanilla normals targets

[alandtse]

Summary

• Restores the CreateRenderTarget_Normals and CreateRenderTarget_NormalsSwap hooks removed in fix: deferredcomposite texture rubberbanding #2178.
• Vanilla SSAO renders correctly again for users without Screen Space GI installed.
• Closes strange shadow-like artifacts #2267

Root cause

DeferredCompositeCS.hlsl:333 writes vanilla-encoded normals through UAV1 (normals.UAV in Deferred::DeferredPasses) so the post-pass vanilla SSAO chain (ISSAORawAO -> ISSAOComposite) sees current-frame normals. Those hook structs added D3D11_BIND_UNORDERED_ACCESS to kNORMAL_TAAMASK_SSRMASK and its swap, which is what makes that UAV non-null.

#2178 removed them because the PS-variant deferred composite from #2150 didn't need that UAV. #2150 was later reverted (1e3b6faa7), restoring the CS path, but the hook removal stayed. Result: UAV1 is null, the CS write is silently dropped, vanilla SSAO samples uninitialized data, and produces hard wedge-shaped shadow artifacts.

The bug only surfaced for users without SSGI installed. The Screen Space GI feature force-disables vanilla SSAO every frame at src/Features/ScreenSpaceGI.cpp:710-712 regardless of its own enabled state, so any rig with SSGI installed never saw the broken output.

Why just restore the hooks instead of git revert fcc5921dd

The other two hunks of #2178 (DeferredCompositePS.hlsl signature fixes and a Deferred.cpp SetupRenderTarget(normalRoughnessRT, ...) removal) were tied to #2150's PS path. Both went away with the #2150 revert -- DeferredCompositePS.hlsl no longer exists and normalRoughnessRT isn't a thing in HEAD. The hook removal in src/Hooks.cpp is the only surviving piece, so a targeted re-add is functionally identical to a revert and reads more clearly in the log.

Diagnosis evidence

RenderDoc capture of an affected frame (event IDs from the bug capture):

• DeferredCompositeCS dispatch at EID 47449: UAVs = [Main(63), NULL, MotionVectors(92)] — UAV1 is ResourceId(0).
• Vanilla kNORMAL_TAAMASK_SSRMASK (resource 82) usage timeline: Clear@41356, PS_Resource@47484, PS_Resource@47821, PS_Resource@47910 — never written, only sampled.
• ISSAORawAO at EID 47821 reads resource 82 as t1 and produces wedge-banded AO into the SAO chain, ending up in SAOTex (resource 154) consumed by ISSAOComposite at EID 47910.

Test plan

• Build ALL preset, deploy to a Skyrim install that does not have Screen Space GI installed.
• Confirm vanilla SSAO no longer produces hard wedge-shaped shadow artifacts (the regression reported by users).
• Capture a RenderDoc frame and verify DeferredCompositeCS dispatch now shows a non-null UAV1 bound to kNORMAL_TAAMASK_SSRMASK, and that the target is written each frame (not just cleared).
• Confirm Screen Space GI installs (with EnableVanillaSSAO=false) still look identical -- SSGI's runtime patch zeroes the vanilla SSAO enable byte before this UAV is ever sampled, so it should be unaffected.
• Confirm Screen Space GI installs with EnableVanillaSSAO=true now produce a correct vanilla SSAO contribution rather than the previous artifact.
• VR sanity check (changes affect VR address tables identically -- third arg 0x5B0 / 0x5C3).

🤖 Generated with Claude Code

Summary by CodeRabbit

• Refactor
  • Enhanced internal graphics rendering infrastructure with improved render target processing and handling for better rendering efficiency.

[coderabbitai]

📝 Walkthrough

Walkthrough

Two new render-target hook thunks for normals creation (CreateRenderTarget_Normals and CreateRenderTarget_NormalsSwap) are added to intercept render-target creation, call ModifyRenderTarget(), and forward to the original function. Installation code registers these hooks in the hook chain.

Changes

Normals Render-Target Hooks

Layer / File(s)	Summary
Hook Thunk Definitions src/Hooks.cpp (lines 589–613)	Two new hook structs CreateRenderTarget_Normals and CreateRenderTarget_NormalsSwap are defined, each intercepting render-target creation, invoking globals::state->ModifyRenderTarget(...), and forwarding to the original function.
Hook Installation src/Hooks.cpp (lines 980–981)	Two stl::write_thunk_call installations are added to Hooks::Install() to register the new normals-related hooks at their respective relocation offsets within the hook chain.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• fix: misc fixes #1474: Modifies render-target hook/thunk installation and handling; related via shared hook system updates.
• fix: deferredcomposite texture rubberbanding #2178: Directly modifies the same CreateRenderTarget_Normals and CreateRenderTarget_NormalsSwap hooks in src/Hooks.cpp.

Suggested reviewers

• doodlum
• soda3000

Poem

🐇 Two normals thunks hop into place,
Intercepting targets with grace,
ModifyRenderTarget's call chimes in,
Forwarding onward—the chain begins!
Hook by hook, the graphics align. ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and specifically identifies the main change: restoring UAV binding for vanilla normals render targets in a deferred rendering context, which directly matches the PR's primary objective.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

No actionable suggestions for changed features.

[github-actions]

✅ A pre-release build is available for this PR:
Download