Skip to content

fix: mesh external emittance in interiors#2294

Merged
alandtse merged 6 commits into
community-shaders:devfrom
Dlizzio:external-emittance-interiors
May 9, 2026
Merged

fix: mesh external emittance in interiors#2294
alandtse merged 6 commits into
community-shaders:devfrom
Dlizzio:external-emittance-interiors

Conversation

@Dlizzio
Copy link
Copy Markdown
Contributor

@Dlizzio Dlizzio commented May 8, 2026

Fixes black effect meshes flagged to use external emittance in interiors.

If it is an interior, and no record defining emittance is set, external emittance is disabled on the mesh.

Summary by CodeRabbit

  • New Features

    • Suppress external emittance indoors when appropriate to improve lighting realism.
  • Improvements

    • Centralized and refactored ambient and directional lighting calculations for more consistent results.
    • Lighting selection now uses scene lighting color when suppression is active.
  • Bug Fixes

    • Minor shader/header stability and guard fixes to prevent parsing issues.
  • Version

    • Updated release version to 3.0.3.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 8, 2026

Review Change Stack
No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 7c3bf177-046b-4cc7-bfdf-11e85df775e2

📥 Commits

Reviewing files that changed from the base of the PR and between cd70057 and 8000151.

📒 Files selected for processing (1)
  • src/Utils/ExternalEmittance.h
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/Utils/ExternalEmittance.h

📝 Walkthrough

Walkthrough

Adds conditional suppression of external emittance indoors: new permutation flag, shader lighting helper refactor, effect shader conditional use of scene lighting, C++ helpers to update permutations, hook integration, and a version bump.

Changes

External Emittance Suppression

Layer / File(s) Summary
State and Shader Flag Definitions
src/State.h, package/Shaders/Common/Permutation.hlsli
SuppressExternalEmittance flag added at bit (1 << 5) to State::ExtraShaderDescriptors and Permutation::ExtraFlags; Permutation.hlsli include guard/trailing region repaired.
Shader Lighting Helper Refactor
package/Shaders/Common/ShadowSampling.hlsli
Adds MinDirectionalLightMultiplier, LightingSampleNormal, ImageBasedLightingNormal and helper functions (GetRawAmbientLighting, GetAmbientLighting variants, GetDirectionalLighting, GetSceneLightingColor); refactors ExtractLighting.
Effect Shader Emittance Suppression
package/Shaders/Effect.hlsl
GetLightingColor uses SuppressExternalEmittance + interior state to replace external-emittance source with ShadowSampling::GetSceneLightingColor().
C++ Suppression API
src/Utils/ExternalEmittance.h, src/Utils/ExternalEmittance.cpp
New ExternalEmittance namespace with ShouldSuppress and UpdatePermutation to evaluate and set/clear the suppression bit on a render pass permutation.
Hook Integration
src/Features/LightLimitFix.cpp, src/Hooks.cpp
Calls to ExternalEmittance::UpdatePermutation(pass) added to BSEffectShader_SetupGeometry thunks after original setup invocation.
Packaging / Version
features/Light Limit Fix/Shaders/Features/LightLimitFix.ini
Version bumped from 3-0-2 to 3-0-3.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Possibly related PRs

Suggested reviewers

  • alandtse
  • doodlum
  • jiayev

Poem

🐰 A rabbit in the shader glade hops light and shade between each blade,
Indoors the glow is gently tamed, a new small flag has been named,
Helpers hum and functions sing, scene-lit color takes the ring,
Hooks align to set the bit, so emittance dims where walls are lit,
Hooray — the lamp is softly stayed. ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: fixing how external emittance is handled for meshes in interior cells, which is the core objective described in the PR description.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 OpenGrep (1.20.0)

OpenGrep fatal error (exit code 2): [00.13][ERROR]: Error: exception Unix_error: No such file or directory stat src/Utils/ExternalEmittance.h
Raised by primitive operation at UTmp.replace_named_pipe_by_regular_file_if_needed in file "libs/commons/UTmp.ml", line 145, characters 8-27
Called from Scan_CLI.replace_target_roots_by_regular_files_where_needed.(fun) in file "src/osemgrep/cli_scan/Scan_CLI.ml", lines 1086-1087, characters 19-65
Called from List_.fast_map in file "libs/commons/List_.ml", line 81, characters 17-20
Called fro


Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 8, 2026

No actionable suggestions for changed features.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/Features/LightLimitFix.cpp (1)

565-572: ⚠️ Potential issue | 🔴 Critical

Critical vtable slot conflict: only one BSEffectShader_SetupGeometry hook can be active.

Both LightLimitFix::Hooks::BSEffectShader_SetupGeometry::thunk and EffectExtensions::BSEffectShader_SetupGeometry::thunk write to the same virtual function slot (0x6) on RE::VTABLE_BSEffectShader[0]. The second hook installation completely overwrites the first, causing one feature's functionality to be silently disabled.

  • src/Features/LightLimitFix.h line 229: stl::write_vfunc<0x6, BSEffectShader_SetupGeometry>(...)
  • src/Hooks.cpp line 961: stl::write_vfunc<0x6, EffectExtensions::BSEffectShader_SetupGeometry>(...)

Additionally, EffectExtensions::BSEffectShader_SetupGeometry::thunk updates globals::state->permutationData.EffectRadius, which the LightLimitFix version does not perform. Resolve this by implementing hook chaining or merging the functionality into a single hook handler.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/Features/LightLimitFix.cpp` around lines 565 - 572, Two hooks are
clobbering vtable slot 0x6 for BSEffectShader_SetupGeometry so only one runs;
fix by merging their logic into a single handler and install only that handler
via stl::write_vfunc<0x6,...>. Create/replace the thunk for
BSEffectShader_SetupGeometry so it calls the original func(This, Pass,
RenderFlags), then calls ExternalEmittance::UpdatePermutation(Pass), updates
globals::state->permutationData.EffectRadius with the same logic found in
EffectExtensions::BSEffectShader_SetupGeometry::thunk, and finally invokes both
feature entry points (LightLimitFix singleton:
BSLightingShader_SetupGeometry_Before and _After and EffectExtensions behavior
if any) in the correct order; remove the second stl::write_vfunc<0x6,...>
install so only the merged handler is written to slot 0x6.
🧹 Nitpick comments (1)
src/ShaderTools/ExternalEmittance.h (1)

1-7: 💤 Low value

Consider adding a forward declaration for RE::BSRenderPass.

The header uses RE::BSRenderPass* without including the necessary type definition. While this works because all current includers already have the RE headers, adding a forward declaration would make the header self-contained.

♻️ Suggested improvement
 `#pragma` once

+namespace RE
+{
+	class BSRenderPass;
+}
+
 namespace ExternalEmittance
 {
 	bool ShouldSuppress(const RE::BSRenderPass* a_pass);
 	void UpdatePermutation(const RE::BSRenderPass* a_pass);
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/ShaderTools/ExternalEmittance.h` around lines 1 - 7, The header declares
functions ShouldSuppress and UpdatePermutation that take RE::BSRenderPass* but
doesn't forward-declare RE::BSRenderPass; make the header self-contained by
adding a forward declaration for namespace RE and class BSRenderPass (so the
declarations of ShouldSuppress and UpdatePermutation compile without requiring
includers to already include RE headers).
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@src/Features/LightLimitFix.cpp`:
- Around line 565-572: Two hooks are clobbering vtable slot 0x6 for
BSEffectShader_SetupGeometry so only one runs; fix by merging their logic into a
single handler and install only that handler via stl::write_vfunc<0x6,...>.
Create/replace the thunk for BSEffectShader_SetupGeometry so it calls the
original func(This, Pass, RenderFlags), then calls
ExternalEmittance::UpdatePermutation(Pass), updates
globals::state->permutationData.EffectRadius with the same logic found in
EffectExtensions::BSEffectShader_SetupGeometry::thunk, and finally invokes both
feature entry points (LightLimitFix singleton:
BSLightingShader_SetupGeometry_Before and _After and EffectExtensions behavior
if any) in the correct order; remove the second stl::write_vfunc<0x6,...>
install so only the merged handler is written to slot 0x6.

---

Nitpick comments:
In `@src/ShaderTools/ExternalEmittance.h`:
- Around line 1-7: The header declares functions ShouldSuppress and
UpdatePermutation that take RE::BSRenderPass* but doesn't forward-declare
RE::BSRenderPass; make the header self-contained by adding a forward declaration
for namespace RE and class BSRenderPass (so the declarations of ShouldSuppress
and UpdatePermutation compile without requiring includers to already include RE
headers).

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 9c8d3124-bb39-4487-b5df-da5c8ee83769

📥 Commits

Reviewing files that changed from the base of the PR and between 4294716 and 1606639.

📒 Files selected for processing (8)
  • package/Shaders/Common/Permutation.hlsli
  • package/Shaders/Common/ShadowSampling.hlsli
  • package/Shaders/Effect.hlsl
  • src/Features/LightLimitFix.cpp
  • src/Hooks.cpp
  • src/ShaderTools/ExternalEmittance.cpp
  • src/ShaderTools/ExternalEmittance.h
  • src/State.h

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 8, 2026

✅ A pre-release build is available for this PR:
Download

Copy link
Copy Markdown
Collaborator

@alandtse alandtse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shadertools is deprecated and shouldn't be used for anything. It's only for historical reasons. Can this be properly placed in the utils directory or some other area? If this should go into commmonlib, also consider putting it there.

@Dlizzio
Copy link
Copy Markdown
Contributor Author

Dlizzio commented May 8, 2026

Shadertools is deprecated and shouldn't be used for anything. It's only for historical reasons. Can this be properly placed in the utils directory or some other area? If this should go into commmonlib, also consider putting it there.

Moved to utils. This doesn't seem low level enough to include in commonlib.

Comment thread src/Utils/ExternalEmittance.h Outdated
@alandtse alandtse merged commit c18daf8 into community-shaders:dev May 9, 2026
11 checks passed
@alandtse alandtse linked an issue May 9, 2026 that may be closed by this pull request
alandtse added a commit to alandtse/open-shaders that referenced this pull request May 9, 2026
Catches up with 4 commits from upstream/dev:
  * refactor(hlsl): use Game constants (community-shaders#2298)
  * fix: mesh external emittance in interiors (community-shaders#2294) -- adds
    Utils/ExternalEmittance, hooks LLF's BSEffectShader_SetupGeometry
    via ExternalEmittance::UpdatePermutation(Pass)
  * fix(grass-collision): avoid sorting actor smart pointers (community-shaders#2299)
  * fix(ishdr): gate bloom for legacy weather mods (community-shaders#2297)

Conflicts resolved:
  * features/Inverse Square Lighting/Shaders/Features/InverseSquareLighting.ini
    Took upstream's 1-3-0 (their refactor; we hadn't bumped ISL).
  * features/Light Limit Fix/Shaders/Features/LightLimitFix.ini
    Kept our 3-1-0 -- supersedes upstream's 3-0-3 because our UI
    consolidation is the larger structural change. Upstream's
    external-emittance hook is folded in alongside.
  * src/Features/LightLimitFix.cpp
    Kept BOTH include sets: our Deferred.h plus upstream's
    Menu/ThemeManager.h and Utils/ExternalEmittance.h. The
    BSEffectShader_SetupGeometry hook gained
    ExternalEmittance::UpdatePermutation(Pass) cleanly via the
    auto-merger -- no manual reconciliation needed there.

Build verified: ALL preset compiles cleanly post-merge.
alandtse added a commit that referenced this pull request May 9, 2026
Catches up with 4 commits from upstream/dev:
  * refactor(hlsl): use Game constants (#2298)
  * fix: mesh external emittance in interiors (#2294) -- adds
    Utils/ExternalEmittance, hooks LLF's BSEffectShader_SetupGeometry
    via ExternalEmittance::UpdatePermutation(Pass)
  * fix(grass-collision): avoid sorting actor smart pointers (#2299)
  * fix(ishdr): gate bloom for legacy weather mods (#2297)

Conflicts resolved:
  * features/Inverse Square Lighting/Shaders/Features/InverseSquareLighting.ini
    Took upstream's 1-3-0 (their refactor; we hadn't bumped ISL).
  * features/Light Limit Fix/Shaders/Features/LightLimitFix.ini
    Kept our 3-1-0 -- supersedes upstream's 3-0-3 because our UI
    consolidation is the larger structural change. Upstream's
    external-emittance hook is folded in alongside.
  * src/Features/LightLimitFix.cpp
    Kept BOTH include sets: our Deferred.h plus upstream's
    Menu/ThemeManager.h and Utils/ExternalEmittance.h. The
    BSEffectShader_SetupGeometry hook gained
    ExternalEmittance::UpdatePermutation(Pass) cleanly via the
    auto-merger -- no manual reconciliation needed there.

Build verified: ALL preset compiles cleanly post-merge.
@Dlizzio Dlizzio deleted the external-emittance-interiors branch May 10, 2026 01:44
ParticleTroned pushed a commit to ParticleTroned/skyrim-community-shaders that referenced this pull request May 15, 2026
Branch-preserving adaptation: keep this branch's existing ExtraShaderDescriptor bit layout, add SuppressExternalEmittance as a new bit, and wire the suppression through the current effect-shadow/IBL path.
ParticleTroned pushed a commit to ParticleTroned/skyrim-community-shaders that referenced this pull request May 16, 2026
Branch-preserving adaptation: keep this branch's existing ExtraShaderDescriptor bit layout, add SuppressExternalEmittance as a new bit, and wire the suppression through the current effect-shadow/IBL path.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Spider webs no longer illuminated in 1.5.1

2 participants