Skip to content

fix(emat): detect bad format textures#2032

Merged
alandtse merged 16 commits into
devfrom
emat-detect-bad-textures
Mar 31, 2026
Merged

fix(emat): detect bad format textures#2032
alandtse merged 16 commits into
devfrom
emat-detect-bad-textures

Conversation

@doodlum
Copy link
Copy Markdown
Collaborator

@doodlum doodlum commented Mar 30, 2026

Detects _m textures which have been saved with multiple channels. Ignores the w channel in detection because sometimes transparency is baked into it.
The math is a bit messy because due to compression the channels are not truly the same.

bugged
ScreenShot27

fixed
ScreenShot28

Fixes #1969

Summary by CodeRabbit

  • Bug Fixes
    • Broadened envmap handling to apply to additional material modes for more consistent reflections.
    • Replaced mip-based mask test with a direct env mask sample and added a guard against improperly formatted env mask textures.
    • Ensured parallax, shadow/mip, and UV selection use the corrected env mask test.
    • Split complex-material color sampling so parallax-updated UVs are sampled correctly.
    • Unified env-mask base sampling and modulation to avoid redundant lookups and visual inconsistencies.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Mar 30, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Broadened the compile-time gating for the EMAT envmap path to ENVMAP || MULTI_LAYER_PARALLAX || EYE. Replaced a high-mip .SampleLevel(..., 15).w test with a non-mipmap Sample(...).w mask, added kMaskEpsilon and pairwise x/y/z equality guards, introduced envMaskBase, and updated parallax/branching and envmask modulation accordingly.

Changes

Cohort / File(s) Summary
Shader Lighting Logic
package/Shaders/Lighting.hlsl
Broadened complexMaterial gating to `ENVMAP

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Suggested reviewers

  • jiayev
  • alandtse

Poem

🐰✨ I hop through pixels, keen and bright,
I sample masks where channels light.
If RGBs align and formats stray,
I quiet parallax and keep misreads away.
A tiny rabbit patch — shaders hummed just right.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check ✅ Passed The code changes directly address the linked issue #1969 by implementing detection of bad textures (specifically _m masks with multiple channels) that were causing visual artifacts on Elven Shield and Shield of Ysgramor.
Out of Scope Changes check ✅ Passed All changes in Lighting.hlsl are directly related to implementing the bad texture detection feature to fix the visual artifacts reported in issue #1969; no unrelated modifications are present.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Title check ✅ Passed The title 'fix(emat): detect bad format textures' accurately summarizes the main change: detecting improperly formatted _m textures with multiple channels.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch emat-detect-bad-textures

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Automated formatting by clang-format, prettier, and other hooks.
See https://pre-commit.ci for details.
@github-actions
Copy link
Copy Markdown

No actionable suggestions for changed features.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@package/Shaders/Lighting.hlsl`:
- Around line 1087-1095: The current detection samples envMaskTest via
TexEnvMaskSampler.SampleLevel(..., 15) and then uses its RGB to decide if the
texture is single-channel, but LOD 15 is over-averaged; change the logic to
sample RGB at a high-detail mip (e.g., SampleLevel with LOD 0) for the
RGB-similarity check while retaining the existing LOD 15 sample for the alpha
threshold test: call TexEnvMaskSampler.SampleLevel(SampEnvMaskSampler, uv, 0)
into a new local (e.g., envMaskDetail) and use envMaskDetail.x/y/z for the
abs(...) comparisons, keeping envMaskTest.w (from the LOD 15 sample) for the
complexMaterial alpha checks.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 9cd5a222-5bee-4e00-bf18-6932c263484d

📥 Commits

Reviewing files that changed from the base of the PR and between d3b20bc and 25e217f.

📒 Files selected for processing (1)
  • package/Shaders/Lighting.hlsl

Comment thread package/Shaders/Lighting.hlsl Outdated
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Mar 30, 2026

✅ A pre-release build is available for this PR:
Download

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@package/Shaders/Lighting.hlsl`:
- Line 1089: The HLSL scope operator is incorrect in the SampleBias call causing
compile failure; replace the single colon usage with the C++/HLSL scope
resolution operator so the call uses SharedData::MipBias; edit the line invoking
TexEnvMaskSampler.SampleBias (referencing SampEnvMaskSampler, uv and
SharedData::MipBias) to use SharedData::MipBias instead of SharedData:MipBias so
the shader compiles.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 21b102a9-c844-4fb7-9205-47015260005a

📥 Commits

Reviewing files that changed from the base of the PR and between 25e217f and 66c23e1.

📒 Files selected for processing (1)
  • package/Shaders/Lighting.hlsl

Comment thread package/Shaders/Lighting.hlsl Outdated
Automated formatting by clang-format, prettier, and other hooks.
See https://pre-commit.ci for details.
@doodlum
Copy link
Copy Markdown
Collaborator Author

doodlum commented Mar 30, 2026

@coderabbitai review

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Mar 30, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

doodlum and others added 2 commits March 30, 2026 17:07
Automated formatting by clang-format, prettier, and other hooks.
See https://pre-commit.ci for details.
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@package/Shaders/Lighting.hlsl`:
- Around line 1086-1114: When EnableComplexMaterial is false the code leaves
envMaskBase at 1.0 which disables ordinary envmask modulation; sample the env
mask unconditionally (e.g. call TexEnvMaskSampler.Sample(SampEnvMaskSampler, uv)
into envMaskSample before or outside the EnableComplexMaterial branch) and set
envMaskBase = envMaskSample.x in the non-complex-material path (while keeping
the existing complexMaterial branch and its logic for envMaskSample and
complexMaterialColor intact) so regular envmask modulation is preserved when
SharedData::extendedMaterialSettings.EnableComplexMaterial is false.
🪄 Autofix (Beta)

❌ Autofix failed (check again to retry)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 94634b99-93a7-4916-a0ab-55b9160997c5

📥 Commits

Reviewing files that changed from the base of the PR and between c3593f5 and 66b7234.

📒 Files selected for processing (1)
  • package/Shaders/Lighting.hlsl

Comment thread package/Shaders/Lighting.hlsl Outdated
@doodlum doodlum changed the title fix(emat): detect bad textures fix(emat): detect bad format textures Mar 30, 2026
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Mar 30, 2026

Note

Autofix is a beta feature. Expect some limitations and changes as we gather feedback and continue to improve it.

Autofix skipped. No unresolved CodeRabbit review comments with fix instructions found.

@alandtse alandtse merged commit 48ba35f into dev Mar 31, 2026
21 checks passed
ParticleTroned pushed a commit to ParticleTroned/skyrim-community-shaders that referenced this pull request May 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Elven Shield and Shield of Ysgramor texture issue

3 participants